Image acquisition planning systems and methods used to generate information for structures of interest

ABSTRACT

The present disclosure relates to improvements in systems and methods in acquiring images via a imaging devices, where such imaging devices can be configured, in some implementations, with an unmanned aerial vehicle or other vehicle types, as well as being hand-held. Images are acquired from the imaging devices according to capture plans where useful information types about a structure of interest (or objects, items, etc.) can be derived from a structure image acquisition event. Images acquired from capture plans can be evaluated to generate improvements in capture plans for use in subsequent structure image acquisition events. Capture plans provided herein generate accurate information as to all or part of the structure of interest, where accuracy is in relation to the real-life structure incorporated in the acquired images.

This application claims priority to U.S. Provisional Application No.62/684,273, filed Jun. 13, 2018, the disclosure of which is incorporatedherein in its entirety.

STATEMENT OF GOVERNMENT INTEREST

This invention was made with government support under contract number1519971 an 1632248 awarded by the National Science Foundation. TheGovernment has certain rights to the invention.

FIELD OF THE DISCLOSURE

The present disclosure relates to improvements in systems and methods inacquiring images via imaging devices, where such imaging devices can beconfigured, in some implementations, with an unmanned aerial vehicle orother vehicle types, as well as being hand-held. Images are acquiredfrom the imaging devices according to capture plans where usefulinformation types about a structure of interest (or objects, items,etc.) can be derived from a structure image acquisition event. Imagesacquired from capture plans can be evaluated to generate improvements incapture plans for use in subsequent structure image acquisition events.Capture plans provided herein generate accurate information as to all orpart of the structure of interest, where accuracy is in relation to thereal-life structure incorporated in the acquired images.

BACKGROUND OF THE DISCLOSURE

Generation of useful information about structures in a scene or otherobjects, items, etc. is a significant area of investigation in computervision research today. With regard to structures specifically, it can bedesirable to image structures to obtain useful information related tothe structure, such as 3D reconstructions, 3D digital representation,inspection results, measurements, construction estimates, and insuranceunderwriting and adjustment, among many others. As would be appreciated,the usefulness of information obtained about structures of interest canbe substantially enhanced when the information type derived therefrom isaccurate, for example quantitatively or qualitatively accurate, inrelation to all or part of the structure.

One significant application for imaging of structures is acquisition ofimages of a structure via an unmanned aerial vehicle, also called a“drone,” that is equipped with at least one imaging device. One exampleof such an application is the generation of images of a roof via anunmanned aerial vehicle from which relevant information about the roofand parts of a roof, as well as the surrounding structures and areasthat might be of interest. This roof information can be included in aroofing report, such as disclosed in U.S. Pat. No. 8,670,961, thedisclosure of which is incorporated herein in its entirety by thisreference.

To successfully generate images from which useful structure informationcan be derived, the unmanned aerial vehicle must be navigated to thelocation where the structure of interest is, navigate through and aroundthe structure, as well as the scene, to acquire a set of images fromwhich the desired information about the structure, for example, a roof,can be generated, followed by navigation of the unmanned aerial vehicleback to a site. Such navigation can be accomplished using a flight plan,also referred to herein as a “capture plan,” that controls navigationand operation of the unmanned aerial vehicle during a structure imageacquisition event.

One such flight plan generation methodology is disclosed in U.S. Pat.No. 9,612,598, the disclosure of which is incorporated herein in itsentirety by this reference. The '598 patent describes methods forgenerating a flight plan and communicating it to the UAV. The disclosedmethodology includes factors relevant to the UAV flight that areprovided by the user or the system. Factors that can be relevant to theflight plan can include the structure footprint, structure height, andgeographic location. These flight plan-related factors are allidentified and incorporated into the flight plan prior to thecommencement of the UAV flight. Accordingly, there is no provision offeedback (other than potentially obstacle detection) during the flight,nor is analysis of the results performed after the flight to determinethe effectiveness of the flight plan. Other capture plan frameworks forgenerating images from which roof information can be generated aredisclosed in U.S. Pat. Nos. 9,805,261, 9,805,261, 10,012,735 and USPatent Publication Nos. US2019/0118945, US2019/0042829, the disclosuresof which are incorporated herein in their entireties. These latter citedpatent documents also do not appear to incorporate at least variationsof feedback during or after image processing to provide analysis of thenature and quality of the images acquired during the structure imageacquisition event, nor do they address improvements in subsequentcapture plans as a direct or indirect result of such acquired imageanalysis.

It is known that a flight pattern of an unmanned aerial vehicle caninfluence the quality of image acquisition during a unmanned aerialvehicle flight. For example, inaccurate centering of unmanned aerialvehicle angle to the structure on orbit or an incomplete grid patterncan generate less dense point clouds that can limit the completeness ofinformation derivable therefrom. Inconsistent flight speed can result inlow image overlap, especially on turns. Operation of the unmanned aerialvehicle at too fast a speed can result in poor overlap and blurring,which can also negatively affect the quality of information derivablefrom the images. Flying too high or too low or incomplete flightpatterns can also result in less than optimum image information.

Problems associated with navigation and operation of a unmanned aerialvehicle during the acquisition of images of a structure from whichuseful information is intended to be obtained can result in the desiredgoal for the image acquisition—that is the reason for which the imagingis being conducted in the first place—not being met in a structure imageacquisition event. For example, some of the acquired images can includeocclusions that can reduce the quality of a 3D reconstruction of thestructure from image processing. Moreover, the acquired images may begood for one type of use, such as providing inspection-levelinformation, but the images may be totally useless for another use, suchas to provide accurate measurements for the structure. Because theacquired images may be only be processed after the image acquisitionevent is completed, it may not be known that the images obtained of thestructure will be unfit for the intended purpose until well after theimage acquisition event is completed.

Currently, a solution for such unsatisfactory image acquisition is torecommend that a second image acquisition event be conducted to generateimproved images. This not only adds to the cost to generate informationabout a structure such as a roof from imaging information, delays canresult in a contractor not gaining a construction job or a homeownerwaiting a long time for an insurance adjustment on her damaged roof. Itis also possible that access to the site where the structure is locatedcan only be obtained once, which means that a failed image acquisitionevent can prevent the acquisition of any information about thestructure. Moreover, to date, methods to improve capture plans byanalysis of images have been ad hoc in nature, in that a subsequentstructure image acquisition event is conducted to generate images thatare better than the images captured in the first image acquisitionevent, without also developing a systematic approach to generatingimages that are, in fact, better in a subsequent image acquisitionevent.

Aspects not directly related to image acquisition via the unmannedaerial vehicle can also affect the ability to obtain the desired levelof structure information. For example, the image sensor may be dirty orthe unmanned aerial vehicle may acquire images when the environment istoo dark. The sensor may not have the appropriate resolution to acquireimages of the desired level of information: a low resolution sensor isunlikely to provide very accurate measurements of a structure becausethe level of necessary detail will not be obtainable from the imageinformation. According to current state of the art, the impacts of suchproblems will typically not be discernible until after the images areprocessed. The desired goal for the image acquisition event maytherefore not be obtainable.

While unmanned aerial vehicles are a well-recognized way today togenerate images of structures, images can also be generated by a personmoving an imaging device around a structure, object, item etc., or bymovement of a non-aerial vehicle equipped with an imaging device througha scene. The success of such image acquisition events is also greatlyinfluenced by the manner in which the images of the structure, etc. ofinterest are acquired, as well as the characteristics of the imagingdevices themselves.

There remains a need for improvements in image acquisition plans fromwhich desired information about a structure or object of interest can bederived in accordance with a desired capture plan goal or target. Thepresent disclosure provides this and other improvements.

SUMMARY OF THE DISCLOSURE

The present disclosure provides improvements in capture plans that canbe generated from after-the-fact analyses of derived structure, object,item, etc. information and the acquired images generated in an imageacquisition event. In particular, the present disclosure can providedeterminations of whether and why or why not that a capture plan goalwas or was not achieved from the implemented capture plan. Such analysescan provide knowledge about the characteristics and effectiveness of thecapture plan used during one or more image acquisition events inallowing the capture plan goal to be met in another image acquisitionevent. To this end, knowledge can be applied to generate and implementcapture plans in one or more subsequent image acquisition events toimprove the nature and quality of structure, item, object, etc.information derivable in that event. Still further, the presentdisclosure can provide improvements in capture plan generation andimplementation while an image acquisition event is underway so as tobetter ensure that the capture plan goal will be achieved from the imageacquisition event before it ends. The present disclosure can alsoprovide predictions of whether a proposed capture plan will allow acapture plan goal to be achieved when implemented in an imageacquisition event.

In one aspect, among others, a method comprises defining, by a computeror a user, a capture plan goal for an aerial imaging event, wherein thecapture plan goal is configured to provide one or more definedinformation types about a structure of interest, and wherein the one ormore defined information types are generated via aerial imaging of thestructure of interest by an unmanned aerial vehicle configured with atleast one image capture device; generating, by the computer or the user,a first capture plan configured to substantially complete the captureplan goal, wherein the first capture plan comprises instructionsconfigured for operating of the unmanned aerial vehicle, wherein theinstructions are associated with operating the unmanned aerial vehicleand navigating, by the computer or the user, the unmanned aerial vehicleto, around, and back from a location proximate to the structure ofinterest; acquiring, by the unmanned aerial vehicle, a plurality ofimages of the structure of interest in a first structure imaging event,wherein the plurality of acquired images are acquired by the unmannedaerial vehicle during the first structure imaging event according to:vehicle operation instructions, vehicle navigation instructions, andimage acquisition instructions; and processing, by the computer, theplurality of acquired images to generate information types about thestructure of interest, wherein the generated information types comprisesat least some of the one or more information types about the structureof interest defined by the capture plan goal. In one or more aspects,the method can further comprise comparing, by the computer or the user,each of the generated information types with the one or more definedinformation types defined by the capture plan goal; and/or determining,by the computer or the user, whether some or all of the generatedinformation types substantially align with each of the one or moreinformation types defined by the capture plan goal.

In various aspects, the one or more defined information types cancomprise one or more of: a 3D representation of all or part of thestructure of interest, wherein the 3D representation comprises a 3Dreconstruction or a point cloud; measurements of all or part of thestructure of interest; counts of the structure of interest or parts ofthe structure of interest; identification of the structure of interestor parts of the structure of interest; orientation of two objects on ornear the structure of interest with respect to each other in a scene inwhich the structure of interest is located; identification of materialsincorporated in the structure of interest; and/or characterization of acondition state for the structure of interest or parts of the structureof interest. The first capture plan can incorporate informationcomprising at least some of: information about the structure of interestknown prior to the image acquisition, wherein the known structureinformation comprises one or more of: estimated dimensions of all orpart of the structure of interest, GPS location of the structure ofinterest, estimated height of the structure of interest, estimated outerboundaries of the structure of interest, and obstacles proximate to thestructure of interest; unmanned aerial vehicle and image capture deviceinformation comprising one or more of: image capture device lensresolution, unmanned aerial vehicle battery life, inertial measurementsensors and associated componentry, unmanned aerial vehicle GPS statusor interference during the image acquisition, altitude of the unmannedaerial vehicle during the image acquisition, temperature data, andunmanned aerial vehicle clock data; number of images to be acquiredduring the first structure imaging event; number of images to beacquired per unit time during the first structure imaging event; numberof images to be acquired per unit of distance traveled by the unmannedaerial vehicle during the first structure imaging event; distancesbetween the unmanned aerial vehicle and all or part of the structure ofinterest during the image acquisition; view angle derivable from anacquired image of the structure of interest or structure part and acorresponding surface or surface part; angle of triangulation derivablefrom each of two points in two images of the same structure of interestor structure part; structure sample distance (“SSD”) between theunmanned aerial vehicle and the structure of interest or structure partduring the image acquisition; ground sample distance (“GSD”) between theunmanned aerial vehicle and the structure of interest or structure partduring the image acquisition; speed at which the unmanned aerial vehicleis to be moving in the scene or environment during the imageacquisition; and/or number of passes to be made by the unmanned aerialvehicle in and around the structure of interest or parts of thestructure during the image acquisition.

In some aspects, the method can further comprise generating informationabout: resolution of the plurality of acquired images; presence orabsence of occlusions in the plurality of acquired images; potentialerror range of information derived from the plurality of acquiredimages; information associated with weather and illumination around thestructure of interest during the image acquisition; orientation of theimaging device with respect to sunlight direction during the imageacquisition; unmanned aerial vehicle gimbal position and stabilityduring image acquisition; obstructions proximate to the structure ofinterest during the image acquisition; and/or acquired imagecharacteristics associated with navigation of the unmanned aerialvehicle, wherein the acquired image characteristics result at least inpart from operation of the unmanned aerial vehicle according to thefirst capture plan. The unmanned aerial vehicle operations can compriseone or more of: the degree of alignment of the unmanned aerial vehiclewith the all or part of the structure of interest during the imageacquisition; the degree of overlap between the acquired imagesincorporating an interior of the structure of interest and imagesincorporating one or more boundaries of the structure of interest; thedegree of centering of the unmanned aerial vehicle relative to thestructure of interest during the image acquisition; degree of forwardand side overlap between the acquired images when the first capture planis configured to acquire images in a grid pattern relative to thestructure of interest; degree of overlap of image radii between acquiredimages when the first capture plan is configured to acquire images in acircular pattern relative to the structure of interest; yaw of theunmanned aerial vehicle during the image acquisition; and/or orientationof the at least one image capture device relative to the structure ofinterest during the image acquisition.

In one or more aspects, the one or more defined information types cancomprise one or more measurements of the structure of interest, andgenerated roof dimensions can be within about 5% of actual roofdimensions when the actual roof dimensions are directly measured. If oneor more of the generated information types do not substantially conformto the one or more defined information types defined by the capture plangoal, the method can further comprise generating a second capture planincorporating information derived from processing of the plurality ofacquired images from the first structure imaging event, wherein thesecond capture plan is used in a second structure imaging event. Astructure of interest in the second imaging event can be the same as thestructure of interest in the first imaging event, or can be differentfrom the structure of interest in the first imaging event.

In various aspects, the method can further comprise generating feedbackabout whether all or part of the capture plan goal has been achieved inthe first structure imaging event, wherein the feedback is provided tothe computer or to the user, and wherein the feedback is optionally usedin the generation of a second capture plan. The feedback can compriseinformation about one or more of the following: view angle derivablefrom an acquired image of the structure of interest or structure partand a corresponding surface or surface part; angle of triangulationderivable from each of two points in two images of the same structure ofinterest or structure part; ground sample distance (“GSD”) between theunmanned aerial vehicle and the structure of interest or structure partduring the image acquisition; and/or structure sample distance (“SSD”)derivable for the unmanned aerial vehicle and the structure of interestor part during the image acquisition. The 3D reconstruction can begenerated. The 3D reconstruction can incorporate all or part of thestructure of interest, Information associated with the 3D reconstructioncan be provided, wherein the provided information comprises one or moreof: point cloud density when the 3D reconstruction comprises a pointcloud; re-projection error measurement for the 3D reconstruction; and/oraccuracy indication for the 3D reconstruction, wherein the accuracyindication is provided in the form of a probability or percentage thatthe 3D reconstruction is an accurate representation of all or part ofthe structure.

In some aspects, the 3D reconstruction can comprise a wireframe, whereinthe wireframe can comprise all or part of the structure of interest. Themethod can further comprise evaluating the wireframe to identify missingor occluded areas; and/or analyzing the plurality of acquired imagesfrom which the wireframe was derived to provide information associatedwith a diagnosis of one or more reasons for the presence of the missingor occluded areas. The method can further comprise incorporating theprovided information associated with the diagnosis in a second captureplan. Instructions can be optionally provided for imaging of at leastpart of the structure of interest from ground-level. At least some ofthe image processing can be conducted during the first structure imagingevent, and/or at least some feedback can be incorporated in the vehicleoperation instructions, the vehicle navigation instructions, or theimage acquisition instructions, thereby allowing modification of atleast some of the first capture plan during the first structure imagingevent.

In another aspect, a method comprises defining, by a computer or a user,a capture plan goal for an aerial imaging event, wherein the captureplan goal is configured to provide one or more defined information typesabout a structure of interest, and wherein the one or more definedinformation types are generated via aerial imaging of the structure ofinterest by an unmanned aerial vehicle configured with at least oneimage capture device; generating, by the computer or the user, a firstcapture plan configured to substantially complete the capture plan goal,wherein the first capture plan comprises instructions configured foroperating of the unmanned aerial vehicle, wherein the instructions areassociated with operating the unmanned aerial vehicle and navigating, bythe computer or the user, the unmanned aerial vehicle to, around, andback from a location proximate to the structure of interest; acquiring,by the unmanned aerial vehicle, a plurality of images of the structureof interest in a first structure imaging event, wherein the images areacquired by the unmanned aerial vehicle during the first structureimaging event according to: vehicle operation instructions, vehiclenavigation instructions, and/or image acquisition instructions;processing, by the computer, the plurality of acquired images togenerate information types about the structure of interest, wherein thegenerated information types comprises at least some of the one or moredefined information types about the structure of interest defined by thecapture plan goal; generating a second capture plan for use in a secondstructure imaging event, wherein: the structure of interest imaged inthe first structure imaging event and a structure of interest imaged inthe second structure imaging event are the same or different, and/or anoutput of each of the first and second structure imaging events is a 3Dreconstruction of the structure of interest imaged in the first or thesecond structure imaging event; and/or comparing each of a first 3Dreconstruction generated from the first structure imaging event and asecond 3D reconstruction generated from the second structure imagingevent with information associated with an associated real-lifestructure, thereby providing accuracy information associated with thefirst and second capture plans. The accuracy information associated withthe first and second capture plans can incorporate measurementinformation providing a percent error or confidence level that the firstand second 3D reconstructions have the same features or dimensions asthe associated real-life structure. The accuracy information associatedwith the first and second capture plans can be incorporated into acapture plan used in subsequent structure imaging events.

Additional advantages of the invention will be set forth in part in thedescription that follows, and in part will be apparent from thedescription, or may be learned by practice of the invention. Theadvantages of the invention will be realized and attained by means ofthe elements and combination particularly pointed out in the appendedclaims. It is to be understood that both the foregoing generaldescription and the following detailed description are exemplary andexplanatory only and are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an implementation of the present disclosure relatingto generated results of a capture plan, in accordance with variousaspects of the present disclosure.

FIG. 2 illustrates an example of a process, in accordance with variousaspects of the present disclosure.

FIG. 3 is a block diagram illustrating an example of a machine that canbe used for wireframe verification, in accordance with various aspectsof the present disclosure.

DETAILED DESCRIPTION OF THE DISCLOSURE

The term “substantially” is meant to permit deviations from thedescriptive term that do not negatively impact the intended purpose. Alldescriptive terms used herein are implicitly understood to be modifiedby the word “substantially,” even if the descriptive term is notexplicitly modified by the word “substantially.

The term “about” is meant to account for variations due to experimentalerror. All measurements or numbers are implicitly understood to bemodified by the word about, even if the measurement or number is notexplicitly modified by the word about.

An “image acquisition event” is a period in which a plurality of images,where a plurality comprises two or more images, that include all or partof a structure of interest is acquired for processing and analysisaccording to the methodology herein. An image acquisition event has abeginning, an operational period, and an end.

“Acquired images” means a plurality of images that are acquired duringan image acquisition event, wherein at least some of the pluralityincorporates image information about at least part of the structure ofinterest. The acquired images are generated from one or more imagingdevices in a format that can be processed according to the methodologyof the present disclosure as set out in more detail hereinafter. In someimplementations, the images are acquired by an aerial vehicle where theimaging device from which the images are acquired can be incorporated inthe vehicle and be operational therewith. Such an example of an imagingdevice incorporated with an aerial vehicle is an “unmanned aerialvehicle.” An imaging device can also be operational with other vehicletypes, such as automobiles, etc.

A “capture plan goal” is a goal or target for the image acquisitionevent or, put another way, information types or collection ofinformation types that are intended to be generated as a result ofprocessing of the acquired images generated during the image acquisitionevent. When in the context of generating one or more information typesfrom a structure of interest, such defined capture plan goals caninclude one or more of:

-   -   Generation of a 3D representation of all or part of a structure        of interest, such as a 3D reconstruction or a point cloud,        wherein additional information can be derived from the        reconstruction. A 3D reconstruction of all or part of the        structure of interest can be provided in the form of, for        example, a mesh, orthomosaic image, a wireframe;    -   Generation of measurements of all or part of a structure of        interest. The measurements can have a level of accuracy as        described hereinafter;    -   Counts or numbers of a structure of interest or parts of thereof        (e.g., an entire roof, or the numbers of different planes in the        roof);    -   Condition state (e.g., new, old, damage, type of damage, color);    -   Inspection-level detail of all or part of a structure of        interest;    -   Identification of a structure type or parts of a structure of        interest;    -   Orientation of two objects with respect to each other (i.e.,        topology);    -   Surveillance and/or situational awareness of a scene or one or        more objects, people, or animals of interest in a scene;    -   Counts of one or more people or animals of interest in a scene;    -   Identification of one or more people or animals of interest in a        scene;    -   Comparison between images acquired of a scene or location in a        first image acquisition event and images acquired of the same        scene or location during a second image acquisition event, for        example, to show changes in the scene or changes in structures        in the scene over time;    -   Efficient image acquisition and image processing parameters        (e.g., image acquisition quantity and quality characteristics        vs. the time to transfer such images and the associated        processing time required);    -   Successful navigation of an imaging device through an        environment;    -   Visualization of an environment or structures, objects, people,        animals, etc. in which the imaging device is traveling;    -   Use of information derived from acquired images in augmented        reality (A/R) and virtual reality (V/R) applications; and    -   Use of information derived from acquired images in Building        Information Modeling (BIM) applications.

Applications such as topology, object identification, AR/VRapplications, BIM applications, among others, are disclosed in U.S. Pat.No. 9,904,867, the disclosure of which is incorporated in its entiretyby this reference.

In accordance with the systems and methods disclosed herein, a firststructure image acquisition event may provide information suitable tomeet a first capture plan goal but may not for another capture plangoal. For example, acquired images that are suitable to meet a goal ofinspection-level detail may not include the necessary detail to allowaccurate measurements of the structure to be obtained, as is discussedfurther herein. A first capture plan goal can therefore be generated todefine at least one specific target or goal for a type or types ofinformation to be derived from the first structure image acquisitionevent. A first capture plan can be defined to generate acquired imagesthat can be processed to derive information relevant to achieving thecapture plan goal, for example, inspection-level detail ormeasurement-level detail. The first capture plan goal and/or theassociated capture plan can be analyzed according to the methodologyherein to provide information to indicate whether the capture plan goal,that is, the goal(s) or target(s) desired or intended from the imageacquisition event, have been met.

A “capture plan” for image acquisition by a vehicle, can includeinstructions for vehicle navigation, vehicle operation, and imageacquisition in an around the scene or structure of interest duringgeneration of acquired images in a structure image acquisition event. Inone implementation, a capture plan suitable for use herein canincorporate instructions associated with generation of the acquiredimages by at least one imaging device during the image acquisition eventby an unmanned aerial vehicle. The plurality of acquired images aresuitable for processing to derive information, for example, in the formof one or more information types, about all or part of the structure ofinterest being imaged. In non-limiting examples, the capture plan can beassociated with at least some of:

-   -   Number of images to be acquired during an image acquisition        event;    -   Number of images per unit time (e.g., frames per second);    -   Number of images per unit of distance traveled by the imaging        device (e.g., number of images acquired per 100 ft of distance);    -   Distances (for example, altitude or linear distance) from the        structure or part of the structure from which each of the images        are acquired;    -   View angle between the imaging device and the structure or        structure part being imaged;    -   Speed at which the imaging device is moving in the scene or        environment where the structure of interest is located;    -   Number of passes made in and around structure during image        acquisition;    -   Information about the structure or scene being imaged where such        information can be identified before the image acquisition event        (e.g., approximate dimensions of the structure, GPS location,        structures or obstacles that must be avoided or accounted for        during image acquisition etc.);    -   Imaging device information (e.g., lens resolution, battery life,        sensor information that can allow relevant data to be recorded        or adjusted during image acquisition etc.);    -   Structure sample distance (SSD), as such term is defined        hereinafter;    -   Duration of image acquisition;    -   Structure height and structure boundaries;    -   Image resolution;    -   Time of day (TOD) of image acquisition;    -   GPS status or interference or potential error range during the        structure image acquisition event;    -   Weather and illumination during the structure image acquisition        event;    -   Orientation of the image acquisition device with respect to        sunlight direction during the image acquisition event;    -   Gimbal stability of a vehicle during image acquisition;    -   Occlusion and obstruction that affects image acquisition;    -   Capture pattern/navigation of the vehicle to structure of        interest matching, capture pattern/navigation of the vehicle        boundary overlap, pattern/navigation of the vehicle centering        relative to the structure;    -   Forward and side overlap for capture plan provided in a grid        pattern and radius overlap for capture plan provided in an orbit        pattern;    -   Yaw of vehicle on turning during image acquisition; and    -   Image/sensor orientation during image acquisition.

“Structure Sample Distance” (“SSD”) for a given surface is defined asthe distance between pixel centers measured on that surface and isfunction of camera distance to the surface and focal length of thecamera. Without being bound by theory, the inventors herein currentlyunderstand that SSD and occlusion can operate as previously unrecognizedfactors in acquiring images of a structure that are suitable to generatea desired level of detail so as to provide the intended informationabout the structure as set forth elsewhere herein. Further, multiplesemantic data points can be pertinent to generating the requisiteinformation needed for inspection level detail for some structures ofinterest, such as color variations and surface characteristics (e.g.,imperfections, damage, etc.), that may only be resolvable/identifiablefrom high resolution imagery, where “resolution” refers to the distanceof the camera to a target surface divided by the focal length of thecamera in pixels.

“Ground sample distance” (“GSD”) in a digital photo of the ground fromair or space is the distance between pixel centers measured on theground. For example, in an image with a one-meter GSD, adjacent pixelsimage locations are 1 meter apart on the ground. In the context of thepresent disclosure, GSD can relate to the distance between the unmannedaerial vehicle and the ground when the subject image is acquired. Insome aspects, GSD can be useful to derive a height for a structure ofinterest from images acquired therefrom.

In a first implementation, at least one image acquisition device can beconfigured to generate acquired images of a structure of interest in thescene, where such acquired images are generated by implementation oroperation of a first capture plan. This capture plan can be configuredto provide instructions associated with navigating and operating animage acquisition device through and around a scene or environment thatincludes a structure of interest so as to generate acquired images thatincorporate information or information types about all or part of astructure of interest. Such acquired images can be processed to derivestructure information therefrom, where the nature and quality of suchstructure information can be evaluated against a capture plan goalgenerated for that first image acquisition event. Such derived structureinformation and capture plans associated therewith are discussed in moredetail hereinafter.

In contrast to the UAV flight plan and output methodology described inU.S. Pat. No. 9,612,598, previously incorporated by reference, thepresent disclosure does not only address generation of structureinformation or damage reports from an image acquisition event. Rather,the inventive system and methods herein are configurable to generateflight plans—or more generally, capture plans—that can be associatedwith a plurality of defined capture plan goals. In this regard,inventive capture plan generation and implementations can allow a userto have a purpose or goal for the flight plan of a roof report, as inthe '598 patent. Significantly, however, the inventive methodology canalso allow generation and implementation of capture plans that areconfigurable to allow one or more additional capture plan goals to beachieved in an image acquisition event, as discussed elsewhere herein.

In broad constructs, the present disclosure provides improvements incapture plans that can be generated from after-the-fact analyses ofderived structure information and the acquired images generated fromwhich the structure information is derived in a structure imageacquisition event. In particular, the present disclosure can providedeterminations of whether and why (or why not) a capture plan goal wasor was not achieved from the implemented capture plan. Such analyses canprovide knowledge about the characteristics and effectiveness of thefirst capture plan used during one or more image acquisition events inallowing the capture plan goal to be met in a subsequent imageacquisition event. To this end, knowledge gained from a first captureplan implementation can be deployed to generate and implement captureplans in one or more subsequent image acquisition events to improve thenature and quality of structure information derivable in that event.Still further, the present disclosure can provide improvements incapture plan generation and implementation while a structure imageacquisition event is underway so as to better ensure that the captureplan goal will be achieved from the image acquisition event before it iscompleted. The present disclosure can also provide predictions ofwhether a proposed capture plan will allow a capture plan goal to beachieved when implemented in an image acquisition event. These and otheraspects of the present disclosure will be discussed hereinafter.

The imaging devices used to generate the acquired images can comprise,but are not limited to, digital cameras, smartphone cameras, tabletcameras, wearable cameras, video cameras, digital sensors,charge-coupled devices, and/or the like. LIDAR, depth sensing cameras,and thermal imaging can also be used. The imaging devices can includeknown or determinable characteristics including, but not limited to,focal length, sensor size, aspect ratio, radial and other distortionterms, principal point offset, pixel pitch, alignment, and/or the like.Information associated with such determinable characteristics can beincorporated into acquired image metadata for use in the informationderivable therefrom. As discussed elsewhere herein, the imaging devicescan be integrated in or operational with aerial vehicles, both mannedand unmanned, or other types of vehicles. In further implementations,hand-held imaging devices may be appropriate.

Imaging of a scene and all or part of a structure of interest (orobject, or item, etc.) therein can be commenced when the at least oneimaging device is present at the scene where the structure(s) ofinterest is located. A plurality of images for processing to generateuseful information can be acquired by movement of the imaging devicethrough and around and up and down around the scene or environment inthe proximity of the structure(s) interest in a manner to providesuitably overlapping images of a quantity and quality to allow acquiredimages from which useful information can be derived. Such imageacquisition can be in accordance with one or more capture plans asdiscussed herein.

Information relevant to image acquisition can include, for example,information about the address of the structure, geographic location ofthe structure (e.g., X, Y, Z coordinates, latitude/longitudecoordinates), GPS location, identity of the structure, owner or supplierof the information order (e.g., contractor, insurance company etc.),time of operation, vehicle speed during travel, weather conditionsduring operation, material type, and other information that could berelevant to the quality, content, and use cases for the acquired images,as well as any information derivable therefrom.

In one implementation, the imaging device can be moved through andaround a scene or environment, for example, in the air or on or nearground level, to allow acquired images to be generated of structuresetc. from locations above the ground. Such images can be acquired, forexample, from unmanned aerial vehicles (“UAVs”), where such UAVs cancomprise unmanned aerial vehicles, satellites, weather balloons, or thelike. As would be recognized, the UAV may include one or more imagingdevices configured to acquire images of a scene or environment, and anystructures (or objects, people, animals, etc.) of interest in the scene.The imaging device can be appropriately positioned on the aircraft toallow suitable images to be generated from an aerial flight. Typically,the UAV will include navigational and operational components such as oneor more global positioning system (GPS) receivers, one or morecommunications systems to send and receive information pertinent to theflight, image acquisition etc., one or more inertial measurement units(IMU) and related componentry (e.g., clock, gyroscope, compass,altimeters) so that the position and orientation of the unmannedaircraft can be monitored, remotely directed, recorded and/or storedwith and/or correlated with particular acquired images. An unmannedaircraft can incorporate temperature sensing equipment, recordingequipment, etc. An exemplary unmanned aerial vehicle in the form of aunmanned aerial vehicle that includes an onboard imaging device is theDJI Phantom® 4 Pro.

In a specific, non-limiting, example, acquired images that include allor part of a structure of interest can be generated by navigating andoperating an unmanned aerial vehicle through and around a scene, wherenavigation, operational, and image acquisition instructions for thevehicle are associated with capture plan instructions generated for astructure image acquisition event.

An unmanned aerial vehicle capture plan can be implemented wholly orpartly by a human operator who provides navigational, operational, andimage acquisition instructions to the aerial vehicle during an imageacquisition event, where such human operator-generated instructions canbe transmitted to and from a remote location where the operator isstationed or at the site where the structure is located. When a human isfully or partially involved in completion of the capture plan, he can beassisted via the providing of visual, audible, or haptic signals thatcan suitably prompt him to modify the vehicle path during a flight.

The navigation, operation, and image acquisition associated with aunmanned aerial vehicle capture plan can also be generated whollyautomatically to allow autonomous operation thereof. Instructions forimplementation of the capture plan to allow the vehicle to fly throughand around a scene or environment to generate the acquired imageswithout intervention by a human operated autonomously can be provided tothe unmanned aerial vehicle (or other vehicle type) via transmission ofinstructions associated with the capture plan to the vehicle from aremote device, or via instructions loaded onto hardware associated withthe vehicle prior to start of the image acquisition event.

Still further, one or more of the navigation, operation, and imageacquisition can be conducted both autonomously and via human operation.During an autonomous operation of the unmanned aerial vehicle, a humanoperator can monitor execution of the capture plan visually via imagesor video that are remotely transmitted to the operator via the onboardimage acquisition device or from flight path information that can beshown on his monitor. If the human operator observes that the unmannedaerial vehicle may need manual intervention, for example, to avoid anobstacle or to deal with a weather event or better position forinspection based on current scene information, the operator can overrideautonomous mode and take over operation of the vehicle. Instructionsassociated with directing the unmanned aerial vehicle to implement acapture plan can be provided to the operator to allow generation of theacquired images in accordance with the capture plan. If humanintervention is indicated, information associated therewith can beincorporated in capture plan goal analysis for subsequent use, asdiscussed further herein.

Acquired images from which structure information can be derived can alsobe generated from manned aerial vehicles, such as helicopters,airplanes, and hot air balloons, or the like. A capture plan can beimplemented by the allowing the pilot to self-direct operation andnavigation of the aerial vehicle. Still further, a capture plan can beimplemented by providing the pilot with operational and navigationalinstructions that direct him to implement the capture plan. Suchinstructions can be visual, audible, haptic, or a combination thereof.Alternatively, instructions can be provided that enable, in all or inpart, automatic operation of the aircraft.

Captures taken from space outside of the earth's atmosphere, can betaken by satellites, or manned and unmanned vehicles. Capture plans forsuch implementations can be implemented autonomously if the vehicle isunmanned or one or more of autonomously or pilot-directed if the vehicleis manned.

Acquired images from which structure information can be derived can begenerated from at or near ground level. For example, an imaging devicecan be held by a person or placed onboard a vehicle, and the person orvehicle can move through and around a scene or environment according toinstructions associated with a capture plan in order to generate theacquired images for processing according to the methodology herein. If aperson is holding the imaging device at or near ground level to generatethe acquired images, audible, visual, or haptic instructions can beprovided to the person to assist in implementing the capture plan. Ifthe imaging device is being moved around the scene or in an environmentat or near ground level by a vehicle (e.g., a car, truck, etc.) thevehicle can be in communication with a remote device that can suitablytransmit operational and navigational instructions to a driver of thevehicle or to the vehicle itself to implement the capture plan.

A capture plan can also be self-directed by a person holding an imagingdevice during generation of the acquired images or when the person isoperating a vehicle (e.g., unmanned aerial vehicle, UAV, car, etc.)unassisted by computer implemented navigational controls. Informationabout self-directed capture plans can be analyzed in accordance with thepresent disclosure.

Still further, acquired images can be generated from underground orunderwater scenes or environments via operation of a suitable vehicle ordevice that can be manned or unmanned. The imaging device associatedwith such vehicle can be moved in and around the scene or environmentaccording to a capture plan that provides navigation and imageacquisition instructions to the vehicle and the imaging deviceassociated therewith.

In broad implementations, the present disclosure relates to automatic,or substantially automatic, analysis of structure information derivedfrom acquired images via image processing of a plurality of images,where such acquired images are generated from digital imagery of a scenehaving one or more structures of interest therein. The capture plan usedto generate the images can be evaluated after the imaging information iscomplete to determine why a capture plan goal may not have beenachieved, among other things. In some implementations, the capture plancan be effectively evaluated while the images are being acquired.

The methods and systems of the present disclosure can be configured togenerate and provide feedback about how well (or not well) one or morestructure information types derived from images acquired fromimplementation of a capture plan match or align with at least onedefined capture plan goal. Conformity to or alignment with the captureplan goal for the structure information type(s) derived from theacquired images is the purpose for which the image acquisition event wasundertaken in the first place. As such, the inventors herein havedetermined that it is not enough to acquire images of a structure ofinterest in an image acquisition event and deriving structureinformation therefrom, that information must be useful for its intended,or defined, purpose.

In some implementations, a capture plan goal can comprise or beassociated with the generation of one or more accurate 3Dreconstructions of a scene or all or part of one or more structures ofinterest in the scene, and subsequent structural and/or semanticanalysis thereof.

The capture plan goal analysis systems and methods of the presentdisclosure can have utility in at least four scenarios or use cases:

-   -   analysis of a capture plan after-the-fact;    -   use of such analysis to generate and implement an improved        capture plan for a subsequent image acquisition event;    -   review and correct a capture plan while an image acquisition        event is underway; and    -   generating predictions of whether a proposed capture plan will        be effective to achieve a desired capture plan goal.        Depending on the context in which the disclosed methodology is        being used, one or more use cases can be applicable.

In a first use case, the present disclosures can have utility inanalysis of a capture plan goal after an image acquisition event iscompleted wherein acquired images are generated from a capture plan thatdirects navigation and operation of an imaging device traveling throughand among a scene or an environment. In a non-limiting illustration ofthis use case, after a structure image acquisition event is completed,an analysis can be performed to determine whether and to what extent thestructure information types derivable from the acquired images meets oraligns with the capture plan goal for that image acquisition event.Feedback regarding meeting or alignment of the structure informationcontents derived from the acquired images with the capture plan goaldesired from the image acquisition event can be provided. As anon-limiting example, several feedback parameters can be calculatedincluding view angle, angle of triangulation, andStructure-Sample-Distance (SSD):

-   -   if extrinsic camera parameters are denoted by C, R, and t where        C is camera center, R is camera rotation, and t is camera        translation, the view angle to a surface with a normal vector of        {right arrow over (n)} will be

${{camera}\mspace{20mu}{plane}\mspace{14mu}{normal}\mspace{20mu}\overset{\rightarrow}{v}} = {{norm}\ \left( {\begin{bmatrix}{R_{20} + t_{0}} \\{R_{21} + t_{1}} \\{R_{22} + t_{2}}\end{bmatrix} - \begin{bmatrix}C_{0} \\C_{1} \\C_{2}\end{bmatrix}} \right)}$${{{view}\mspace{14mu}{angle}} \propto} = {\cos^{- 1}\left( {\overset{\rightarrow}{n} \cdot \overset{\rightarrow}{v}} \right)}$if calibration matrix for the first camera is denoted by K₁ andcalibration matrix for the second camera is denoted by K₂, thetriangulation angle between homogeneous coordinates of two correspondingpoints (p₁ and p₂) on the first and second image can be calculated asp ₁ ^(n) =K ₁ ⁻¹ *p ₁ and p ₂ ^(n) =K ₂ ⁻¹ *p ₂q ₁ =R ₁ *p ₁ ^(n) and q ₂ =R ₂ *p ₂ ^(n)triangulation angle β=cos⁻¹(q ₁ ·q ₂)if focal length of a camera in pixels is denoted by f and the distancebetween the camera center to a 3D point on a given surface is denoted bydSSD=d/f

The systems and methods herein can be configured to return qualitativeinformation as feedback related to the degree of which desired orintended structure information type, such as set out in the capture plangoal, was derived from the acquired images that were generated from acapture plan. Still further, the systems and methods can be configuredto return quantitative information associated with the correctness oraccuracy (or not) of the derived imaging information. In this regard,information can be provided that provides a confidence in an accuracylevel for the information obtained. A qualitative measurement can bederived from the processing of the acquired images, whereby suchmeasurement can be denoted as “good” or “bad” or “high confidence” or“low confidence” or “suitable” or “not suitable” other similardenotation. For example, a structure being imaged during an imageacquisition event can have an actual height of x feet when measureddirectly, that is, by hand. Structure information can be derived fromthe acquired images that indicates that the returned information, whichcan include a height assessment, can suitably be used for a purposedefined in the capture plan goal where the goal is something other thanmeasurement accuracy. Such uses can include generatingnon-mathematically accurate 3D reconstructions of that structure. Inthis regard, the capture plan goal of non-mathematically accuratestructure measurements can be achieved from such image acquisitionevent, and the capture plan used to generate the acquired images can bedeemed as suitable. Alternatively, it may be determined that theacquired images were not suitable to generate a non-mathematicallyaccurate 3D reconstruction of the structure, because processing andanalysis of the acquired images indicates that at least the heightelement of the structure cannot be suitably estimated in relation to theother features of the structure. In this case, if the capture plan goalwas to be able to generate a 3D reconstruction having a high level ofqualitative accuracy, the capture plan goal will not be achieved. Such anon-mathematically accurate capture plan goal can comprise inspectionlevel detail in some implementations.

Alternatively, a numerical measurement can be derived from the acquiredimages and that measurement information provided with quantitative levelof the accuracy for that measurement, such as a percent error betweenthe derived structure measurements and the actual measurement of thestructure or part of the structure that was imaged during the imageacquisition event when such measurements are generated directly from thestructure in real life. For example, the structure being imaged duringan image acquisition event can have an actual height of x when measureddirectly. Measurements can be derived from the imaging information to aprovide a height of x ft+/−y inches, where y is an estimated measurementerror. If the capture plan goal analysis indicated that the measurementerror is within an acceptable % error for the derived measurement, forexample, not more than 0.1%, the derived height measurements allow thecapture plan goal for the image acquisition event to be achieved.

As a non-limiting illustration, analysis of the imaging information froma completed image acquisition event can be configured to providefeedback associated with the probability that the structure of interest(or parts or portions thereof) was suitably incorporated in the acquiredimages in accordance with a capture plan in a manner that allowed thecapture plan goal to be achieved. Still further, the analysis of thederived structure information in relation to the characteristics of theacquired images can provide feedback related to a deviation or lack ofalignment from the capture plan goal caused, at least in part, by thecontent of the capture plan used to generate the images.

As would be appreciated, not all capture plan goals will be the same.Two different capture plan goals can require that at least some of theacquired images from which one or more structure information types arederived comprise some differing characteristics so as to allow thedesired information about the structure to be derived.

Using an unmanned aerial vehicle flight as an example. In some usecases, for example for the purpose of generating measurements of a roof,roofing data can be collected by an unmanned aerial vehicle and imageryis processed into 3D representation of that roof from which the desiredmeasurements can be derived. In other use cases, the vehicle can beoperated to acquire images sufficient to enable a detailed inspection ofthe structure. It is possible that the capture plan used to generateaccurate measurements may differ, at least in part, from the captureplan that will generate a good inspection-level result. In this regard,inspection may require close up imaging of the roof so that smalldefects may be observable (e.g., to determine a condition state, such ashail damage or approximate age), whereas the obtaining of measurementsmay require imaging from a greater distance. To achieve these differinggoals, the respective capture plans can be different in at least somerespects. In other use cases, an unmanned aerial vehicle flight may havethe goals of both measurement and inspection-level details, which canrequire that a capture plan be generated to allow both of these capturegoals to be achieved.

A capture plan goal may be image stitching to generate orthogonalimages, as opposed to 3D reconstruction or measurement of a structure.In this use case, the capture plan can be configured to generate anoverall capture of the structure or a birds-eye capture that cangenerally be without concern for whether one or more of the acquiredimages may comprise some blur that would otherwise be unacceptable forother capture goals.

In a further example, if the capture plan goal is efficiency, such as inimage acquisition time and/or image processing, a capture plan thatgenerates too many images could be problematic. Too many images mayextend the image acquisition time, such as in the flight time requiredfor an unmanned aerial vehicle as well as extend processing and imagetransfer time. A suitable capture plan to address efficiency can begenerated to provide the smaller or minimum number of acquired images ina reduced or shortest image acquisition time but still satisfy thecapture plan goal(s).

In one aspect, the inventive systems and methods can providesuccess-related information that indicates or signals the likelihood, inthe form of a probability assessment or confidence level, of whether thegoals or objectives set out in the capture plan goal were met. Suchsuccess-related information can optionally include information that canincrease the confidence that the one or more structure information typesderivable from the acquired images is correct in context, such asproviding evidence, reasoning, data, metrics, etc. to support thereturned success-related information and any conclusions associatedtherewith.

As a non-limiting example of such feedback, an image acquisition eventcan comprise a unmanned aerial vehicle flight for imaging of a roof of ahouse. A capture plan goal can be provided that indicates thatmeasurements to within about 1% of the actual roof measurement should beachieved from the image acquisition event. A capture plan intended toachieve the selected level of measurement accuracy can be implemented,and images of the roof can be generated therefrom during an imageacquisition event. The acquired images can be processed and thenanalyzed to determine whether they comprise the characteristics requiredto generate not only the desired roof measurements, but also to returnthe range of measurement error specified by the capture plan.

As a non-limiting example of feedback, an image acquisition event cancomprise a unmanned aerial vehicle flight for imaging of a cell tower orsimilar structure. Cell tower imaging and the generation of usefulinformation is described in detail in U.S. Provisional PatentApplication No. 62/729,982, filed Sep. 11, 2018, the disclosure of whichis incorporated herein in its entirety. As a non-limiting example, acapture plan goal can be provided that indicates that measurements towithin 6″ of the actual tower height should be achieved from the imageacquisition event. Additional goals could include within 1″ of theactual RAD height and width, and within 2″ of actual RAD elevation abovea base plane. A capture plan intended to achieve the selected level ofmeasurement accuracy can be implemented, and images of the tower can begenerated therefrom during an image acquisition event. The acquiredimages can be processed and then analyzed to determine whether theycomprise the characteristics required to generate not only the desiredtower measurements, but also to return the range of measurement errorspecified by the capture plan. One example of a means of providing thefeedback or error can include selection and presentation of groundcontrol points (“GCP”) if present in the scene. Presenting imagesshowing the GCPs showing projection accuracy or offset of center pointscan communicate a confidence level in the accuracy results and meetingthe capture plan goals. Scale constraints can also be applied togenerate accuracy information and/or confidence levels.

Still further, feedback or information can be returned that identifiesaspects of the acquired images, and therefore information derived orderivable therefrom, that might have differing levels of error oraccuracy, where such error or accuracy would be introduced as a resultof the characteristics of the acquired images. Such imagecharacteristics can be a result of the capture plan used to generate theimages. In the house and roof examples that were imaged by an unmannedaerial vehicle, a user can be provided with an assessment of what partsof the roof are suitable for a particular use. In this regard, therecould be a 100% probability that the house was suitably imaged inaccordance with the capture plan goal of imaging the house to allowgeneration of a 3D reconstruction of the house that is accurate forgeneration of a non-mathematically accurate model of the house and theroof. However, the user can be provided with only a 50% probability thatthe roof overhang areas were suitably imaged, thereby providing a lesserthan 100% chance that a desired goal of achieving an accuratemeasurement of the roof square footage can be achieved. If the captureplan goal was to generate accurate roof measurements of the total roofarea, the defined capture plan goal will not be achieved.

It can be recognized that this example highlights that such returnedmeasurement information signals that problems in image acquisitionlikely existed as a result of the capture plan used to generate theacquired images from which the measurements were intended to be derived.As discussed hereinafter, information derived from such image andcapture plan analysis can be used to generate improved capture plans toimprove the ability of a user to achieve a desired capture plan goal.

Alternatively, feedback can be returned that it is unlikely that thedesired capture plan goal cannot be achieved. For example, if a desiredgoal for the unmanned aerial vehicle flight is for accuracy within 1 cmand the SSD generated from the unmanned aerial vehicle-generated imagesis 5 cm, there will be no chance that the desired accuracy for thesubject unmanned aerial vehicle flight will be obtained and the captureplan goal will not be achievable. Appropriate feedback of that absolutefailure can be returned. While the information type derivable therefromwill not conform to the capture plan goal for that image acquisitionevent, such information type can be used to improve capture plans inother contexts as discussed further herein.

Various types of image acquisition characterization can be relevant inreturning useful feedback regarding the characteristics of acquiredimages acquired in an image acquisition event. In non-limiting examples,these can include one or more of:

-   -   The presence or absence of image overlap in different directions        and the amount thereof between a plurality of images;    -   The angle of image acquisition device in relation to an        identified/desired surface of an object being captured (i.e.,        view angle). View angle is defined as the angle that a ray        emanating from the camera center intersects with the target        surface. The ideal case is that this ray intersects the surface        at 90 degrees angle because that is when the image is formed the        best;    -   Structure-Sample-Distance (SSD);    -   Ground-Sample-Distance (GSD);    -   Angle of triangulation, where such comprises the angle derivable        between two points on two different images of the same structure        or structure part;    -   Re-projection error and accuracy (or lack thereof) of 3D        reconstruction or image alignment;    -   Camera sensor resolution and/or lens distortion;    -   GPS resolution and noise level thereof;    -   Image texture type and level;    -   Image illumination level;    -   Image noise and blurriness;    -   Existence of occluded regions;    -   Image coverage for occluded regions;    -   Density of the 3D representation of the scene (e.g., density of        the point cloud);    -   Number or interval of acquired sensor observations during the        image acquisition event such as number or interval of images or        number or interval of GPS readings or number or interval of IMU        (“inertial measurement unit”) readings;    -   Image acquisition settings (e.g., aperture, shutter speed, ISO,        time-lapse, FPS for video data);    -   Accuracy of GPS information acquired during an image acquisition        event, as indicated by type/model/manufacturer of GPS sensor;    -   Use of stability sensors or optical image stabilization on        imaging device and/or vehicle (e.g., unmanned aerial vehicle,        UAV, car etc.) during image acquisition; and    -   Characteristics of sensors (e.g., high or low quality) as        indicated by type/model/manufacturer of imaging device        (smartphone, camera) or vehicle (e.g., unmanned aerial vehicle,        UAV, car etc.).

In a further aspect, after processing of the acquired images so as toderive one or more information types relevant to the structure ofinterest, which could include, for example, generating a wireframe or 3Dreconstruction of the structure, additional analysis can be performed onthe acquired images, information incorporated therein or derivabletherefrom, as well as any other information that may be associated withthe acquired images. Such associated information can include, forexample, weather or temperature information or time of day and/orposition of the sun related to the structure that may be relevant togeneration of the acquired images during the image acquisition event. Inthis regard, the system and methods herein are configurable to provideinformation on what steps or actions were conducted when acquiring theacquired images and any other pertinent information used to generate theimages from which structure information was derived.

Upon processing of the acquired images, a wireframe can be generatedtherefrom, for example. This wireframe can be further analyzed to assessthe overall coverage of the structure by the acquired images, such as bydetermining the quality of coverage and occluded areas on the structure.If an absence of coverage and/or occluded image areas are revealed bysuch analysis, it can be inferred that such absences of acquired imagespertinent to the structure of interest resulted from the capture planused to generate the acquired images.

For example, generation of a 3D reconstruction of the structure ofinterest could have included extending a wireframe into occluded areaspresent in the acquired images. Still further, the 3D reconstructioncould have been generated by extending edges to intersect in areas inwhich there was no associated point cloud derivable from the acquiredimages, for example, because the images included occluded areas, missingareas, or for other reasons. Still further, illumination informationcould be absent from the structure image information, which can affectthe type of information derivable from the captured information. If a 3Dreconstruction of the structure of interest was generated whenpotentially pertinent information was absent from the acquired images,it can be inferred that the information derived therefrom may notinclude the characteristics necessary to achieve a specified captureplan goal. Moreover, recognition that pertinent information may bemissing from the acquired images could indicate that the capture plan bywhich the images were generated was not accurate or complete.

Analysis of the acquired images and the structure information derivedtherefrom can help provide a diagnosis of the reasons for the absence ofrelevant information needed to achieve the capture plan goal desiredfrom an image acquisition event. That is, failure to achieve a specifiedcapture plan goal, and how to rectify that failure, comprise newinsights provided by the inventors herein. Such insights can be used todevelop improvements in image acquisition, for example, by generatingimprovements in capture plans as discussed herein after. Such insightscan also be useful to improve image processing so as to enhance thenature and quality of structure information derivable from an imageacquisition event that is conducted in subsequent capture plandeployment.

Appropriate feedback can be provided to a user that the capture plangoal and associated capture plans did not provide the desired coverageof the scene or environment and any structures (or objects, people,animals, etc.) located therein, and that any images acquired during theimage acquisition event may not allow the defined capture plan goal tobe achieved. Moreover, such analysis and associated feedback can alsoprovide information regarding the accuracy (or lack thereof) of thederived structure information.

Using aerial imaging of a house and roof as an example, when a userviews the pre-processed output of the house and roof images on hisscreen, he might subjectively believe that the camera has covered thehouse and roof sufficiently well because it will be difficult, if notimpossible, to visually observe image acquisition errors that mightnegatively affect the ability to derive the intended information aboutthe structure of interest. In other words, a human supervisor of animage acquisition event may be unlikely to be able to determine visuallyin real time or substantially in real time that the acquired imagesgenerated according to the capture plan being implemented cannot besuitably processed to derive structure information that allows thecapture plan goal to be achieved. As a result, resources expended forthe image acquisition event could be wasted, and any capture plan goals,such as the generation of a cost estimation report for a customer basedon information derived from the acquired images, may not be achieved.

Still further, the qualitative or quantitative accuracy with which awireframe was derived from captured image information that includes thestructure or object of interest can be provided. For example, awireframe may be projected or estimated from such image information, andthis wireframe can be compared to or compared with previously confirmedor validated acquired images for the object of interest. This can allowa wireframe generated from images captured in a second image acquisitionevent to be referenced to a wireframe generated in a first imageacquisition event. In this regard, one or more areas of a generatedwireframe that are associated with reconstructions having a lowconfidence of qualitative or quantitative accuracy can be identified viafeedback by such a validation or confirmation step. Such areas of lowconfidence can also be stored as potentially relevant to generation andimplementation of the capture plan that resulted in the generation of awireframe having low confidence. Comparison of the first capture planand the second capture plan, along with analysis of acquired images andstructure information respectively associated therewith, can be used togenerate improvements in capture plans as discussed elsewhere herein.

Information associated with whether or not a capture plan goal was metin an image acquisition event can be incorporated into instructionspertinent to subsequent image acquisition events. For example, ifanalysis of the acquired images indicates that one or more features oraspects of a first capture plan was implicated as a reason why a captureplan goal was not achieved, instructions associated with a secondcapture plan can be modified to provide improvements.

Areas of a generated wireframe or plurality of wireframes from which a3D reconstruction can be generated can be visually identified for auser, such as by coloring or otherwise indicating in an observable way.Yet further, confidence levels in the presented 3D reconstruction, andany information derivable therefrom (e.g., measurements, identification,topology, etc.) can be quantified or qualified as feedback for a user orfor incorporation into instructions associated with operation of thecapture plan or for use in subsequent capture plans. For example, for anoccluded area present in the acquired images that were generated from acapture plan, a confidence level in the 3D reconstruction of thatoccluded area can be provided, for example, at least 75% confidence thatthe structure of interest (or part of the structure of interest) wasaccurately reconstructed. For other areas of a 3D reconstruction thatmay have differing levels of occlusion or other imaging characteristicsthat may reduce the accuracy of a 3D reconstruction, a confidence levelfor that area of the structure may be different, such as being higher orlower. Accordingly, the present disclosure can optionally provide one ora plurality of reconstruction accuracies for a 3D reconstruction of allor part of a structure of interest. In this regard, one reconstructionaccuracy can provide an overall reconstruction accuracy for thestructure or part of the structure, whereas a plurality ofreconstruction accuracies can provide accuracies for various parts ofthe reconstruction of the structure or structure part.

The reconstruction accuracies can be presented qualitatively orquantitatively as discussed elsewhere herein. A user can then beprovided with not only a qualitative report as to what part or parts ofa structure may not have been accurately reconstructed from the acquiredimages, but also a quantitative report as to the overall accuracy of thereconstruction, as well as the accuracy of reconstruction for specificparts of the object.

For example, the overall accuracy of a 3D reconstruction of a houseimaged according to a unmanned aerial vehicle capture plan can bedetermined to be of high qualitative accuracy (e.g., “good” or“acceptable”), but a specific part of the house reconstruction, such asone or more areas of roof overhang where occlusions may be present onthe acquired images, may be of low qualitative accuracy (e.g., “poor” or“unacceptable”). Alternatively, the user can be provided with feedbackthat the quantitative quality of a 3D house reconstruction as thedesired information type to be derived was high (e.g., 95% or greaterreconstruction accuracy), but that the quality of reconstruction of oneor more areas on the house near a roof overhang may be of lowerquantitative quality (e.g., 50% or less confidence level ofreconstruction accuracy). Such qualitative and/or quantitative accuracyinformation can be used to determine whether one or more aspects of thecapture plan used to generate the acquired images from with the 3D housereconstructions were suitable to achieve the capture plan goal. If not,such information can be used to generate improvements in capture plangeneration and implementation, as discussed elsewhere.

The acquired images generated from a capture plan wherein all or part ofa structure of interest is imaged during an image acquisition event canbe processed in a variety of contexts to generate useful feedback andconclusions as to the effectiveness in achieving the capture plan goal.In non-limiting examples of such processing categories, one or more ofthe following image-related characteristics can be analyzed from theacquired images (with other examples present in the disclosure herein):

-   -   Point cloud density;    -   Point cloud completeness with no holes or non-reconstructed        areas as related to the structure of interest;    -   Reconstruction accuracy and reprojection error;    -   Occlusion;    -   Point cloud noise;    -   Camera pose error;    -   Accuracy of ground control points;    -   Presence or absence of image overlap in different directions and        the amount thereof;    -   Angle of image acquisition device in relation to an        identified/desired surface of an object being captured (i.e.,        view angle);    -   Structure sample distance;    -   Angle of triangulation;    -   Re-projection error and accuracy (or lack thereof) of 3D        reconstruction or image alignment;    -   Camera sensor resolution and/or lens distortion;    -   GPS resolution and noise level thereof;    -   Image texture type and level;    -   Image illumination level; and    -   Image noise and blurriness.

The systems and methods of the present disclosure can provide a userwith feedback associated with quality of the 3D reconstruction of astructure of interest as derived from acquired images, where suchfeedback may include direction that the user perform a second imageacquisition event to improve image acquisition. Moreover, in somenotable aspects, the present disclosure provides feedback of whatcircumstances could have caused the image acquisition to fully orpartially fail so that a capture plan goal could not be achieved. Inthis regard, a capture plan generated for a second capture event can beconfigured to reduce or eliminate those characteristics or situations inimage acquisition that influenced or caused problems in the first imageacquisition event.

Such identification of one or more problematic image acquisition aspectsor characteristics that occurred in a first image acquisition event andthe reduction or elimination of such problematic aspects in a secondimage acquisition event can be generated automatically by creating a newcapture plan for the second (or subsequent) image acquisition event orby modification of the first capture plan to include corrections asindicated from failure analysis of the output of the first imageacquisition event. In this regard, correction of the second capture plancan be provided substantially without user interaction.

For example, if a first image acquisition event does not generatesuitable imaging information to meet a specified capture plan goal forthe structure of interest because the acquired images include too manyoccluded areas to allow an accurate 3D reconstruction of the structureto be generated, instructions can be incorporated into a second captureplan so that subsequently acquired images for that same structuregenerated in a second image acquisition event are less likely to includesuch occlusions that caused problems with the output of the first imageacquisition event. In this regard, if an overhang on a house is poorlyimaged in the first image acquisition event in which a first captureplan is implemented, the second capture plan can include navigationaland operational instructions for the unmanned aerial vehicle that willfacilitate generation of additional images of at least that area of thehouse, that is, roof overhang areas, where occlusions were evident inthe previously acquired images. Through such correction of the secondcapture plan, it can be expected that there will be fewer or even nooccluded areas in the acquired images of the structure, and that 3Dinformation derivable therefrom can incorporate improved accuracy vs.the 3D information obtainable from the first image acquisition event.Additionally, such knowledge that a house of a particular style, design,location, etc. might be likely to generate occluded areas in imagesgenerated therefrom can be incorporated into a capture plan for adifferent structure with the goal of enhancing the likelihood that animage acquisition event for that structure will result in achieving thecapture plan goal for that structure with only one image acquisitionevent needed. As one example, a capture plan can be generated thatutilizes two different imaging devices to address areas of a structurethat might be difficult to suitably image based on the style or design.A house may require unmanned aerial vehicle image acquisition for theroof and a high resolution camera for upper stories, whereas groundlevel capture could be conducted with a smartphone camera.

Still further, the identification of image acquisition features orcharacteristics that are associated with capture plans and acquiredimages generated therefrom that are likely to or that will cause acapture plan goal to not be achieved can be useful to train users todesign improved capture plans, and therefore execute better on captureplans.

The information derived from capture plan and capture plan goal analysisaccording to the methodology disclosed herein could, over multiple imageacquisition events, lead to the generation of greatly improved captureplans for a structure of interest that can be implemented by either orboth of a computer or user that would reduce the overall failure rate ofimage acquisition events when the desired capture plan goal is notachieved. In this regard, if over multiple image acquisition events,fewer occlusions are generated in the acquired images because analysisof capture plans and capture plan goal success or failures can provideinformation on how subsequent capture plans can be generated andimplemented to reduce the number of occluded areas. Thus, it could beexpected that the 3D information derivable from an image acquisitionevent be improved over time. The present disclosure therefore furtherincorporates systems and methods to improve capture plan generation andimplementation.

Moreover, the methodology of the present disclosure can have utility intraining of pilots who are flying (if a manned flight) or directing theflight (if an unmanned flight), at least by providing them with feedbackin real-time or substantially in real-time to correct or modify acapture plan being implemented by that pilot. This real-time orsubstantially real-time feedback will then not only better ensure thatthe acquired images generated during that image acquisition event willbe more likely to allow the capture plan goal to be achieved, but thefeedback may also be incorporated in that pilot's self-directed captureplans in future image acquisition events.

As a non-limiting example, the real-time or substantially real-timefeedback can be generated using an estimator (i.e., inference model)that has the following characteristics:

-   -   Measurements or observations such as GPS reading, IMU readings,        image acquisition device motion control parameters, etc. that        can be expected to be noise corrupted and the uncertainty in        these measurements or observations can be expected to be        translated to uncertainty in inference;    -   Any a priori information that can be expected to be incorporated        in the estimation;    -   A model might be employed to determine how a system is expected        to evolve over time;    -   It is not expected that the measurements or observations will        always be in a pre-defined coordinate system; and    -   It is expected that the model itself be uncertain.

In view of the above, a goal or objective can be to obtain the bestestimate)? for a parameter x given a set of k measurements orobservations Z^(k)={z₁, z₂, . . . , z_(k)}. The following likelihoodfunction can be defined to represent the conditional probability of themeasurement z given a particular value of x:L≙p(z|x)considering a Gaussian distribution for noise and a normalizing constantn:

${p\left( z \middle| x \right)} = {\frac{1}{n}e^{\frac{- 1}{2}{({z - x})}^{T}{P^{- 1}{({z - x})}}}}$the maximum likelihood estimate {circumflex over (x)}_(m,l) can becalculated as:{circumflex over (x)} _(m,l)=arg max_(x) p(z|x).

The disclosed methodology can also have utility in directing orcorrecting the capture plan of an operator of a terrestrial orunderwater vehicle by providing him with real time feedback of whether acapture plan will generate information about the structure of interestthat will allow the capture plan goal to be achieved. Also, it could beexpected that such feedback will be incorporated in the driver'simplementation of subsequent self-directed capture plans. In thisregard, the disclosed methodology can also include a method to train apilot of an aerial vehicle or a driver of a terrestrial or underwatervehicle to acquire images via a capture plan that is directed towardgenerating images that meet a goal or target for the image acquisitionevent for which the pilot or driver is acquiring images.

In further aspects, the disclosed methodology comprises the performanceof at least two image acquisition events, wherein acquired images of allor part of a structure of interest are generated during each imageacquisition event according to implementation of a generated captureplan, wherein the capture plans for each image acquisition event are notsubstantially identical. Generation and/or implementation of eachdefined capture plan can be by either or both of a computer or a user.The acquired images from each image acquisition event and capture plancan then be processed according to the methodology herein to determinewhether the information types derivable from each set of acquiredimages, respectively, allows a defined capture plan goal to be achieved.The effectiveness of each capture plan to generate the desired captureplan goal for the image acquisition event can be identified so as toallow only that structure information derivable from the better captureplan to be utilized, for example.

Still further, a plurality of capture plans that can be used to generateacquired images of a structure of interest can be ranked as to theireffectiveness in allowing a defined capture plan goal for an imageacquisition event to be achieved. Such insight can be used to select forimplementation of capture plans that can allow a specific capture plangoal to be achieved, while at the same time using only the imagingresources and generating only the number of acquired images needed toachieve the capture plan. For example, analysis of images acquiredaccording to a capture plan may indicate that the capture plan did notprovide acquired images having the characteristics needed to allowaccurate measurements to be derived for the structure of interest. Ifthe capture plan goal for that image acquisition event is to obtainaccurate measurements, that capture plan would fail to provide thedesired result. Thus, selection of that capture plan for use in asubsequent image acquisition event where accurate measurements of thestructure are desired would be improper. However, other useful 3Dinformation about the structure may still be derivable from acquiredimages generated according to that capture plan. If a capture plan goalindicates that less-than-accurate measurements of a structure areindicated, that previous capture plan cane be selected for use.

Such selection of a capture plan based upon the capture plan goal for animage acquisition event can result in a reduction in the amount ofresources needed to generate information about a structure of interest.For example, if only general or qualitative information is needed for astructure of interest, fewer images in fewer locations in and around astructure and an associated scene may need to be acquired. This canreduce the time needed to operate the imaging device, which can bebeneficial when the image acquisition device and the navigational andoperational componentry associated therewith are battery powered. Fewerimaging devices and operators may be needed if imaging time can bereduced. Moreover, generation via an unmanned aerial vehicle of acaptured image set of sufficient quality to allow numerically accuratemeasurements of a structure to be derived may require additional imageacquisition resources and imaging processing time that, in somesituations, may not be desirable to expend.

To illustrate, an accurate 3D reconstruction of a structure may beobtained from acquired images during a first image acquisition event,but the analysis of those acquired images may indicate that the desiredmeasurement accuracy was not derivable from the acquired imagesgenerated from the first capture plan. Instead, to obtain the desiredmeasurement accuracy for the structure, a second image acquisition eventmust be conducted. This will require more unmanned aerial vehicle flightand operator time, as well as more image processing capability. If, inthe future, less than measurement accuracy is desired from an imageacquisition event, the capture plan that generates the desired qualityor quantity of 3D information for the structure of interest can beselected and implemented. If the capture goal is instead measurementaccuracy, the capture plan that did not give the defined goal will notbe used. The elimination of capture plans from a list of capture planoptions via selection (or deselection) thereof can better ensure thatresources can be better utilized and better results can be returned to auser more quickly.

Communication of one or more of the capture plan analysis, imageprocessing results, user feedback, or improvements to be made insubsequent capture plans can be via delivery of information to the user,where such delivered data can be qualitative (e.g., color, shading, heatmaps, sounds, condition state etc.) or quantitative (e.g., measurements,angles, counts etc.). Information can be delivered to a user viawebsite, or printed, or through API, etc. The delivered information caninclude a description of the relevant capture plan(s), the effectivenessof the plan(s) in allowing images acquired therefrom to be suitablyprocessed to meet the capture plan goal for that image acquisitionevent. Suggestions for improvements in capture plan generation andimplementation can also be communicated to the user, or such informationcan be incorporated in instructions associated with generation ofacquired images in an image acquisition event.

Still further, the delivered capture plan analysis information caninclude predictions associated with a relative or absolute accuracy ofderived 3D information for one or more regions of the structure ofinterest as derivable from acquired images generated from a captureplan. For example, an image acquisition event can comprise a definedcapture plan goal of generation of accurate measurements of a roof as aninformation type to be attained via image acquisition by unmanned aerialvehicle. The acquired images can be generated using a capture plan thatcauses the unmanned aerial vehicle to fly in a capture plan intended forreconstruction accuracy, but not measurement accuracy. Such accuratemeasurements may be needed by a contractor or insurance company toprovide information needed to repair or replace the roof, in oneimplementation. If the selected capture plan is known to not provide theintended capture plan goal, accurate measurements will not be obtained.However, the derived 3D information for the roof can still have utilityin roof estimations. Accordingly, the relative accuracy of informationderivable from a capture plan can be provided to a user. The user canthen adjust any estimates provided to a customer to account for theknown lack of accuracy in the 3D information generated from that captureplan, for example, by adding or subtracting a percentage or value fromthe generated numerical information.

Information returned to the user could also provide a quality rating fora capture plan. For example, if the capture plan goal is directed towardgeneration of accurate measurements of all or part of a structure, andthe analysis of the images acquired from a capture plan indicates thatsuch measurements are not derivable therefrom, the capture plan and theassociated image acquisition event can be denoted as “fail” or the like.If the analysis shows that the capture plan goal has been achieved bythe capture plan, the capture plan can be denoted as “success” or thelike. For failures, the structure information derived therefrom can beplaced in a “rejection queue” to prevent its use in downstreamapplications. Similarly, information that meet the capture plan goal canbe placed in an “acceptance queue” for use in downstream applications.The respective capture plans and information associated therewith can bestored for use in generating or analyzing other capture plans andconformance thereof with capture plans, if appropriate.

As indicated elsewhere herein, information that does meet a capture plangoal may nonetheless be suitable for other image acquisition events.Accordingly, a previously rejected capture plan can be selected for usein a current image acquisition event.

Information associated with an image acquisition event or informationderived from the acquired images can be displayed visually to a user inthe form of 2D image or as a 3D tool. Visual display could be in 2Dimages or in a 3D tool, in some implementations. Various userpresentation examples are described in U.S. Patent Publication No.US20180053347A1, the disclosure of which is incorporated herein byreference.

Information derived from the image acquisition event, includinginformation about the structure derived from the acquired images, isconfigurable to be provided to a user in per point or per location onthe structure from which such information is derived. Such derivedinformation can also be presented to the user in summary form, such asin a report or the like. The derived information can also be stored asdata in a form suitable for use in other processing operations in whichthe capture plan, the analysis, and information types derived therefromcan be useful.

The user can be provided with feedback related to the suitability of theselected imaging device to generate a defined capture plan goal. Forexample, the characteristics of images generated from a specific imagingdevice may be unsuitable to meet a specified capture plan, even withgeneration and implementation of an optimum capture plan. While suchsuitability between an imaging device and the capture plan goal can beapplicable to a variety of image acquisition event types and structuresof interest, the specific example of unmanned aerial vehicle imageacquisition via a provided capture plan is illustrative. As would berecognized, the quality of the imaging sensor on the unmanned aerialvehicle can operate as a baseline for the quality of acquired imagesgenerated in an image acquisition event. Therefore, the quality ofimaging information for the structure can be affected by the selectionof an imaging device. In this regard, a defined capture plan goal may begeneration of accurate measurements of a structure of interest as theinformation type to be obtained. However, the imaging sensor on theunmanned aerial vehicle may not comprise the appropriate imagingcharacteristics required to deliver such measurement accuracy when thedefined capture plan instructs the unmanned aerial vehicle to acquirethe images from too far away to allow generating suitable imageresolution. That capture plan goal may nonetheless be suitable for usewith an unmanned aerial vehicle that incorporates a higher resolutionimage sensor. In such a case, the user can be provided with informationabout the mismatch between the specified capture plan goal at the imageacquisition event, and the likely or certain failure of such captureplan goal to be achieved if that unmanned aerial vehicle is used. Aswith other capture plan goal analyses herein, information about therelationship between a certain imaging device and a capture plan goalcan be stored for use.

Still further, the present disclosure can provide feedback as to thequalitative and quantitative characteristics of whether a specificcapture plan did or did not allow a capture plan goal to be achieved, aswell as what information about the structure of interest is derivable asa function of a capture plan. For example, analysis of acquired imagescould deliver feedback such as there is a 100% probability that enoughimages were captured of a target area such as a roof were covered fromimplementation of a capture plan that was intended to generate roofinginformation and/or measurements. However, with the implemented captureplan, feedback could be delivered that there was only a 50% probabilitythat the roof overhangs were successfully imaged. In regards to thesereturned probabilities, feedback could be provided that there is a 95%chance one can reach the capture plan goal of accuracy with aimplementation of a selected capture plan.

Alternatively, feedback could be provided that there is no way that thecapture plan goal can be achieved using a selected capture plan to imagea structure of interest. In this regard, each capture plan can begenerated according to a design or plan that is directed towardeffectively achieving the desired capture plan goal to be implementedfor different structure types, the locations of the structure, theenvironment in which the structure is located, the time of the imageacquisition event, the weather, among other things. For example, acapture plan that is suitable to acquire images of a residential roof ona bright spring day to derive useful information therefrom might be verydifferent from a capture plan that is suitable to acquire images of acellular tower in the winter for inspection. Moreover, as discussed, acapture plan that is suitable to derive reconstruction accuracy may bewholly unacceptable to generate measurement accuracy, even when the samestructure of interest is involved. Feedback can be provided to make surethat a generated capture plan is suitable for the intended capture plangoal.

Also, inappropriate capture plans can be automatically locked out fromuse when a specified capture plan goal cannot be achieved using thatcapture plan. For example, if the SSD for a capture plan cannot allowthe desired accuracy in a selected capture plan goal to be achieved,that capture plan can be removed from use for that and similar imageacquisition events.

Feedback and conclusions that can be implemented to provide usefulinformation about the effectiveness of a capture plan in achieving acapture goal include, but are not limited to (with other examplespresent in the disclosure herein):

-   -   Image overlap amount;    -   Capture device angle with respect to a given/desired surface;    -   Structure sample distance;    -   Angle of triangulation;    -   Re-projection error and accuracy of 3D reconstruction or image        alignment;    -   Camera sensor resolution and lens distortion;    -   GPS resolution and noise;    -   Texture level;    -   Illumination;    -   Image noise and blurriness;    -   Image coverage for occluded regions;    -   Density of the 3D representation of the scene (e.g., density of        the point cloud);    -   Number of observations such as number of images or number of GPS        readings for a point or number of IMU (Inertial Measurement        Unit) for a point, etc.; and    -   Improper capture settings (aperture, shutter speed, ISO,        time-lapse, FPS for video data).

After processing the acquired images, which could include generation ofa wireframe or a 3D representation of the structure of interest,additional analysis on the processed image information to identify anysteps that were used to create or generate the 3D representation forareas where the acquired images may have been missing at least someinformation about an area that is included in the reconstruction. Forexample, an applied step could have included extending a wireframe intoareas that were occluded in the acquired images, or an applied stepcould have involved extending of edges to intersect in areas with nogenerated point cloud, either because occluded areas, missing coverage,or others. The need to apply correction or augmentation steps to the 3Dreconstruction can indicate a lack of completeness and/or coverage froma capture plan that can be improved when the reasons for the problems inthe capture plan are deconstructed by evaluating the acquired imagesgenerated in the first instance by that capture plan.

A user who reviews the generated imaging information prior to processingmight see that acquired images are providing full coverage for thestructure of interest. The flight path of an unmanned aerial vehicle isshown in FIG. 1, as path 105, and the locations relative to the roofstructure where the capture plan instructions provide for acquisition ofimages is shown by circles 110 on path 105. Once the generated pluralityof images are processed, missing information needed to meet a definedcapture plan goal can be indicated by the failure to obtain the desiredinformation type from the acquired images generated in an aerial flight.For example, a wireframe or accurate 3D reconstruction may not beobtained as defined in the capture plan goal. In this event, additionalprocessing of the acquired images can be used to identify aspects of thecapture plan that may have been a cause, or part of the cause, of thecapture plan goal not being met.

As shown in FIG. 1, such areas of concern can be visually identified asareas with missing imaging information or where the characteristics ofthe underlying imaging information result in those areas of the derivedinformation lacks confidence in 3D reconstruction accuracy, for example.In this regard, shading 125 in roof 100 shows areas where “too few”images were obtained, which can also be termed “under-sampling.” Suchunder-sampling can result in a lack of confidence in the informationgenerated therefrom, at least because there may not be enoughinformation derivable from the acquired images to create the desiredqualification or quantification for the structure, here a roof. On theother hand, acquired image information indicated by shading 115 in FIG.1, which can be termed as “over-sampling” of images, or “too many”images acquired in those areas. When more images are acquired than areneeded to derive the information type and accuracy thereof as set out inthe capture plan goal, at least extra processing and storage are neededto provide the desired information. Although the information obtainablefrom such oversampling of images may have a high degree of accuracy whenthe image processing is complete, the capture plan goal may not have hadaccuracy of the generated information as an intended outcome. Thus, suchover-sampling of images is an example of a defined capture plan notbeing met by an image acquisition event. Shading 120 in FIG. 1illustrates areas in the capture plan associated with FIG. 1 that can beconsidered to provide a balance between too few and too many acquiredimages in an unmanned aerial vehicle. In other words, shading 120illustrates areas in where the desired information type has beenachieved, and the defined capture plan has been met in those areas.

Information associated with such over-sampling, under-sampling, and“just right” image acquisition characteristics can be included ininstructions associated with second, or subsequently performed, captureplans. Such information of whether a defined capture plan goal has beenmet (or not met) can be included in instructions associated withsubsequent capture plans used in image acquisition events for the samestructure (i.e., to acquire a second image plurality of images for thesame structure) or to generate a capture plan for use to acquire imagesfor a second structure that is different from the first.

In some implementations, visual feedback associated with a capture plancan be provided as illustrated in FIG. 1, where the path of the aerialvehicle is shown by the arrows. Such feedback can provide shaded as inFIG. 1, colored, labelled, or quantified for presentation to a user.Such information can also be used to highlight potential areas ofconcern in a capture plan.

In some aspects, a capture plan can include information that providesthresholds for image acquisition parameters. For example, and as wouldbe recognized, in order to extract information from a plurality ofimages of a structure or structure part, at least some of the acquiredimages must have a suitable amount of overlap, for example, at leastabout 10 or 20%, or more of less. It follows that generation ofinformation from an image acquisition event should obtain not onlyenough images of a structure or structure part of interest, but alsoenough images that are overlapping. Accordingly, a capture plan that isgenerated for imaging of that structure or structure part can includeinstructions that allow such suitably overlapping plurality of images tobe generated. On the other hand, and as discussed elsewhere herein, acapture plan goal may not require a degree of detail that dictatesacquisition of a large number of images. For example, inspection-leveldetail may typically require fewer overlapping images of the structureor structure part (or object etc.) than measurement detail wouldrequire. As such, a capture plan may be configured to acquire more orfewer images having more or fewer parameters (e.g., view angle, angle oftriangulation, SSD, GSD, etc.), where such parameters are indicated by adefined capture plan goal.

Returning to FIG. 1, shading 120 can then represent a threshold that hasbeen achieved for a specific image parameter, which can also comprise an“information type” defined by a capture plan goal. In contrast, shading115 and 125 can represent areas where acquired images are more or fewerthan the target threshold, and thereby can indicate that a definedcapture plan goal has not been met. In non-limiting examples, thethresholds set for the information types derivable from a capture planillustrated for roof 100 in FIG. 1 were:

-   -   1) Angle of triangulation (threshold=3 degrees);    -   2) Overlap (threshold=4 image views);    -   3) View angle (threshold=30 degrees);    -   4) SSD (threshold=0.5 in.); and    -   5) Measurement accuracy (threshold=0.5 in.).

Still further, for a given occluded point in a 3D reconstruction of astructure where the reconstruction is derived from a set of acquiredimages that include a plurality of occluded areas, a confidence levelfor a first predicted structure area on the reconstruction might be 75%,but another confidence level for a second predicted structure area mightbe 85%. A heatmap can be generated for those regions that might beassociated with a lower confidence in the accuracy of thereconstruction.

In a further broad implementation, the present disclosure also allowssubsequent image acquisition events to be improved, such as byimplementation of better capture plans. When a capture plan is conductedunder similar circumstances in a future image acquisition event, extracare can be given when acquiring images in or at those areas in astructure that were shown as being problematic in the previous imageacquisition event. For example, a user who is implementing a captureplan can be provided with feedback relevant to specific areas of thestructure of interest. This feedback can direct the user to capture aspecific portion of the structure in order to improve the outcome orresolve any issues. The disclosed methodology also can providehighlighting of problem areas (e.g., areas of reconstruction that havelow confidence) to inform the user that there may be a problem with thederived structure information and suggest that a second imageacquisition event be conducted. Still further, the present disclosurecan provide the user with specific information of the cause of suchmissing information and also direct the user on how such an imagingproblem can be rectified in future image acquisition event. Moreover, insome aspects, the disclosed methodology can provide a quantitativeanalysis of the missing image information. For example, a user can beinformed that “5 feet of the area below this roof overhang is partiallymissing from the acquired images,” and the location of that area can bevisually identified for a user. When a capture plan is implemented forthat structure or another structure having similar features in thefuture, the area of concern, here 5 feet of area below the roofoverhang, be subject to extra or more robust imaging.

The methodology of the present disclosure can be useful for trainingpeople how to generate improved capture plans, and to better implementthose improvements when generating images of a structure of interest. Tothis end, the additional processing that can identify why problems mayhave occurred in a structure reconstruction and the generation ofinformation related thereto (e.g., measurements, identifications,topology etc.) can provide capture plan scenarios during training thatcan later be avoided outright because the person has already beentrained to recognize that problems may arise if used in a real imageacquisition event.

Post-processing of generated reconstructions to identify reasons forerrors in the 3D reconstructions can be used when a user is conductingimage acquisition at a location. The user can conduct multiple imageacquisitions while at the location. Processing of each set of acquiredimages onsite to generate reconstructions while at the location,followed by an analysis of the reconstructions to determine whether thecapture plan resulted in incomplete acquired images can allow the userto ensure that he can acquire the right type and amount of acquiredimages in a single trip to the location. If missing acquired images andthe reason thereof are identified by analysis of the reconstructions andassociated image onsite, the user can apply such information to director inform implementation of a capture plan in a subsequent imageacquisition event while on location.

Communication of analysis/results/feedback related to evaluation of thereconstructions and any problems associated with previous and subsequentimage acquisition according to a modified or improved capture plan canbe with visual data or quantitative data, delivered through website, orprinted, or through API, etc. The communication to a user could describethe capture plan, show the effectiveness of the capture plan for meetingthe capture plan goal, as well as communicate improvements to be used insubsequent capture plans. The user can also be provided with informationthat predicts the expected or relative accuracy in structure informationoverall and per structure region if the identified improvements ormodifications are made to the capture plan in a subsequent imageacquisition event. All or part of the visual data can be combined tocreate a health rating for information that has a specific rejectioncriterion.

Visual display of information to a user of information related toreconstruction errors and associated modification to capture plans,among other things, can be in 2D images or with a 3D tool. Visual datapresented to the user in this regard can be as described elsewhereherein. Information related to reconstruction errors and associatedcapture plans can also be presented in summary or other form.

Feedback can also be provided about an imaging device that was utilizedto generate a 3D reconstruction, and any limitations or specificationsthat might affect the generation of reconstruction errors. For example,if the imaging device has only a specific esolution camera, and thedistance between the imaging device and the structure of interest islarge as defined by the capture plan, the image acquisition might appearsuitable, but the image device specifications might be inappropriate inSSD to generate reliable results for the capture plan.

In a further implementation, the methods and systems of the presentdisclosure can allow a capture plan and the associated acquired imagesto be analyzed during an image acquisition event to determine whetherthe defined capture plan goal will be met. Such real-time orsubstantially real-time processing of the acquired images and anyinformation types derived therefrom can be especially useful when theimage acquisition is being conducted without a human operator, forexample, when the image acquisition is done using an autonomous vehicleor a robot. However, the real time or substantially real time processingof acquired images while an image acquisition event is underway can alsobe useful when the acquired images are being generated by novices ornon-expert or even expert operators. The processing of acquired imagesduring an image acquisition event can allow the capture plan, imagescaptured therefrom, and any structure information derived therefrom tobe directed or managed in response to information that is in the processof being generated at that time, and not after an image acquisitionevent is completed.

Feedback resulting from processing of the acquired images during animage processing event can be provided to a user and/or directlyincorporated in the capture plan that is currently underway, as suchuser feedback is described elsewhere herein.

If the images are being captured by a mobile device, and detailedanalysis of the likely outcome for an image acquisition event that iscurrently underway, distance between the imaging device and thestructure can be especially pertinent to the accuracy of thereconstruction of the structure and any information related thereto.Texture can also be relevant to such results and feedback on texturelevels and their impact could result in suggesting a different device tocapture effectively.

The real time or substantially real time processing of images during animage acquisition event has utility in image acquisition via mobiledevices such as smartphones and the like, as well as with unmannedaerial vehicles and other use cases.

Real time machine learning can be applied to information relevant to thestructure of interest or the scene or environment in which the structureis included, or with which it is associated.

In one aspect, the real time or substantially real time processing ofacquired images during an image acquisition event can allow changes tobe made dynamically to a capture plan while image acquisition is stillunderway. For example, instructions can be provided to the navigationaland operational componentry of a vehicle on which the imaging device isconfigured. Some instructions can comprise:

-   -   1) slow down movement of the imaging device if images show poor        texture;    -   2) slow down movement of the imaging device if time of day        signals poor illumination of the scene or structure;    -   3) change viewing angle to better capture occluded areas;    -   4) change movement direction to better capture occluded areas;    -   5) change movement direction where geometric components require        more careful imaging;    -   6) change overall pattern (e.g., add double cross to and orbit        pattern);    -   7) fly lower and slower where hail or other damage condition        detected or predicted; or    -   8) on multi-sensor unmanned aerial vehicles, switch from RGB to        Infrared, switch from RGB to Lidar, etc. Changes to the capture        plan can also be made dynamically based on user or computer        input or feedback from the capture device or other sensor about        conditions of the scene or changes in capture plan goals or        other reasons.

Still further, the present disclosure can provide predictions for theexpected image acquisition effectiveness from a proposed capture planprior to the beginning of an image acquisition event and alignmentthereof with a capture plan. To generate such a prediction of theoutcome of an image acquisition event, including the accuracy or lackthereof any information derivable from acquired images generatedaccording to a capture plan, several inputs can be useful. For example,one or more objectives for the image acquisition event, that is, thecapture plan goal, can be provided as discussed previously. To generatea capture plan that is appropriate for implementation to achieve thedefined capture goal, various information can be provided that may berelevant to the nature and quality of the acquired images, as was alsodiscussed previously.

Generation of a prediction of whether a capture plan goal will be met bya capture plan may be facilitated by additional information such as:

-   -   The scene pulled from outside sources (e.g., Google Earth®,        Google Maps®, satellite data, weather sources);    -   Nearby building information;    -   Material or color of structures;    -   Weather;    -   Premises access rules relevant to the image acquisition event;    -   Lighting and sun path; and    -   Occlusion prediction, etc.

Such provided inputs can allow estimation of the some or all of the samepost-capture metrics that would be present in acquired images. One ormore capture plans can be simulated using information derived fromanalysis of information types derived from previous image acquisitionevents using known inputs relevant to a scene, structure, etc. that isassociated with an upcoming image acquisition event and an associatedcapture plan goal. In this regard, a likelihood of whether or not aproposed capture plan goal can be obtained to better ensure that acapture plan goal for a future image acquisition event can actually beachieved. If the prediction indicates that the capture plan goal cannotbe met from implementation of a capture plan, the proposed capture plancan be rejected. One or more additional capture plan goal predictionscan be generated until a desired prediction is generated.

Still further, a partial set of inputs can be provided, and feedbackprovided of a prediction that a capture goal will be met by a captureplan, although a prediction based on such fewer inputs may have lessconfidence.

Information associated the prediction regarding the capture plan andcapture plan goal can be used to validate or confirm whether theprediction was accurate (or not), and the degree to which accuracyresults. Such validation or confirmation information can be incorporatedto further improve capture plans for later use.

As part of the prediction, a suggestion that can aid in the generationor design of a capture plan that can allow the capture plan goal to bemet can be provided. A suggestion that a specific image acquisitiondevice or combination of image acquisition devices can be generated. Forexample, if the capture plan goal is to capture an entire structure ofinterest, and user inputs or system information determines that largeoverhangs are present on the structure, the system could determine thatcapture from multiple angles is required using a combination ofground-based image acquisition and unmanned aerial vehicle oraerial-generated image acquisition. In a further example, if the capturegoal is a siding estimate, a prediction can be provided of what aspectsof the structure of interest should be imaged from a unmanned aerialvehicle and what aspects should be imaged from a ground-based device,where the ground-based device can comprise a combination of devices,such as a handheld device for lower sections of the structure and ahigher resolution device for upper sections.

In conjunction with the prediction, the system can specify minimum imagedevice requirements, such as camera resolution, type of device, batterylife, etc. For example, generation of the required texture to meet thecapture plan goal might require depth sensing or LI DAR, or the neededangle for the image acquisition might require an unmanned aerial vehicleequipped with a specific camera type. In a further illustration, thepresence of a roof overhang and/or the height of structure walls mightrequire use of a high-resolution camera from ground level. Stillfurther, a time of day may need to be specified for the images to betaken to avoid glare etc. These and other relevant information andactions needed to achieve the desired capture plan goal can be providedto the user to ensure that the predicted result will, in fact, beattained.

In further aspects, when changes or modifications are made to a captureplan for a unmanned aerial vehicle, such changes or modifications can beconfigured to be incorporated information associated with other softwarethat can be used in acquisition of images of a structure, etc. Forexample, information about improvements that can be made in theacquisition of images (e.g., vehicle operation, navigation and/or imageacquisition) that is derivable from a first capture plan and evaluationof whether the goals of that capture plan have been met (or not met) canbe incorporated for use with flight plans associated with products suchas Precision Mapper®, Pix4D Capture®, software included with unmannedaerial vehicles, among others. Moreover, to the extent that the systemsand methods of the present disclosure can generate recommendations formodifications to the settings of image acquisition devices, where suchmodifications are associated with achieving a capture plan, such asshutter settings or resolution, or LI DAR device settings, etc., thesystems and methods herein are configurable to generate suchmodifications in the image devices used to acquire images.

Still further, the systems and methods of the present disclosure canapply a summary rating to a capture plan, based on an analysis of alikelihood of that the capture plan will allow the capture plan goal tobe met. Such a grade can provide a way to compare across imageacquisition events. A summary metric can also provide a way to choosewhether to accept or reject a set of acquired images from an imageacquisition event based on this summary rating.

The systems and methods herein can be operated as pipeline ofapplications or they can be called independently. The system can be madeavailable via an API. In the case of an API, the system can provide acallable endpoint for analysis of acquired images, which would beaccepted as inputs goals, and scene or structure-related information.This can also include capture plan file (if available), image devicefile, resulting 3D representation, and the acquired images. In othercases, the system might incorporate only an acquired image file andinformation about the structure of interest and required quality ofresults, and, optionally, provide the results.

Still further, the system can be configured to characterize the captureplan based on past knowledge of capture plans, such as in case ofunmanned aerial vehicle flights, orbit, grid, double orbit,boustrophedonic, etc. Relating of a new capture plan to a known captureplan can assist in effectively communicating the feedback to a user whois used to a specific output or interface.

The system can be configured to characterize or present the analysis andfeedback herein based on past knowledge of autonomous software programsfrom other vendors, such as Drone Deploy®, Precision Mapper, etc. Suchfeedback could include specific settings to use in setting up autonomousflight software from such other vendors. Change over from another vendormay then be easier, which can be a benefit when a user is trained to usea specific type of software program from which effecting changeover maybe difficult.

The system can be configured to interpret the scene contents and whennecessary provide additional analysis around a structure of interest andin relation to the capture plan goal associated with an imageacquisition event. In the case of inspection for solar estimation andinstallation, a goal for the image acquisition event could be toidentify the height of any existing or potential occlusions on thestructure that may affect solar use, including chimneys and trees.

A user typically wants to know that the images captured in an imageacquisition event are sufficient to meet the objective of the captureplan goal before the imaging device is removed from the scene. Sometimesthis requires premises access permission, or long travel distances,equipment rental, setup time, etc. to capture the images necessary ofthe scene and a structure of interest that are required to generate thedesired results. If the user fails to generate an adequate imageacquisition set while the imaging device is located on the scene, it maybe at least expensive and time consuming if the image acquisition eventdoes not yield the desired results. Moreover, if access cannot beobtained at the scene for a second time, the desired results may beimpossible to obtain.

It is often not trivial to determine on location if the acquired imageswill support goals of the capture plan. For example, if the capture plangoal includes both measurements and inspection as the information typesto be generated from an image acquisition event, each of whichincorporate different information and generate different results, it maynot be possible to determine whether both of these objectives have beenmet while onsite. In short, without first viewing the outputs ofcaptured image processing (including reconstruction/structure frommotion), one cannot know whether the acquired images will conform to thecapture plan.

Moreover, if the acquired images are not processed until later when theunmanned aerial vehicle is away from the scene, and it is determinedthat the capture plan goal was not met, several problems can arise. Forexample, it can be difficult in the first order to analyze a captureplan after the fact. It can also be difficult to determine the reasonsthat poor 3D reconstruction of a structure was generated or low accuracyof measurements were returned. Further, it can be difficult to determinewhether there is low accuracy in the results, without first performingan accuracy study. And, even if the resultant 3D reconstruction appearsacceptable to a human eye, analysis of the 3D representation associatedtherewith can reveal that some areas were not captured sufficiently foraccurate 3D representations of the structure of interest, if that is thecapture plan goal for the image acquisition event. It can also bedifficult to provide feedback to pilots or mobile users in a way to makeit easy to design or fly a subsequent capture plan.

One prior art method provides a way to view image acquisitioneffectiveness. In this regard, Pix4D offers visualization of camerapositions at each capture point. If the user selects a point on a pointcloud, ray tracing can be obtained. This allows lines to be shown fromthe selected point to each camera, which provide information about fromwhich camera the selected point was seen. This, in turn, generatesinformation about what aspects of the images contributed to a 3Dreconstruction of a structure of interest. However, the Pix4D methoddoes not account for occlusion that may be present in acquired images.One could then infer that the confidence of accuracy of that point fromthe number of, and angles of, the cameras used to generate the images ofthe structure contribute to that point in 3D space. There areimprovements needed in this prior art methodology at least because onlyimage coverage is addressed therein, whereas occlusion is ignoredtotally. Errors can therefore be introduced in this prior artmethodology because a camera might be shown as contributing to theselected point when the point cannot, in fact, be seen from that cameraangle.

The systems and methods herein not only shows cameras for every point inthe point cloud, but also limits the points shown to non-occludedcameras, bringing in SSD, formalized camera angle analysis, etc.

Yet further, the prior art provides only limited analysis of acquiredimages after processing to generate the output information about thestructure etc. Such limited information includes, for example,presentation of cameras representations over the scene, rays specifyingfocus area of the camera, and images that stemmed from the camera. Datafor this analysis is often captured in files provided that cameralocations and EXIF data are stored in images. From such informationprior art methods can link images to camera positions as a convenientway to zoom on a structure.

With regard specifically to unmanned aerial vehicle image acquisition,the systems and methods of the present disclosure can be used to improvecapture plans used to acquire images for structures of interest etc. forunmanned aerial vehicles. Such flights can be manual, that is, conductedby an operator who directs the unmanned aerial vehicle navigation usinga controller, user interface, or the like. The unmanned aerial vehiclecan be flown autonomously using navigational instructions provided tothe vehicle. In some cases, the unmanned aerial vehicle can be operatedwith a combination of manual and autonomous navigation.

Some issues that can be relevant to manual flights include:

-   -   Acquiring more images than necessary due to irregular        patterns—longer flight, reconstruction, transfer times;    -   Poor centering of vehicle to building center on an orbital        capture plan so as to generate point clouds that have holes or        missing areas;    -   Speeding up/slowing down on turns—with timer-based capture        logic, results in inconsistent camera placement, poor overlap on        fast legs, too much overlap on slow legs;    -   Going too fast overall—poor forward overlap, blur issues, poor        reconstruction;    -   Multiple flight grid heights—poor scaling; and    -   Too few legs on grid pattern, poor side overlap, poor        reconstruction.

Some issues that can be relevant to autonomous flights can include;

-   -   Flying too high—poor GSD, too many images, long 3D        reconstruction times;    -   Flying too low—low overlap or too many images for a good 3D        reconstruction time;    -   Pattern Selection—choosing orbit with circles, when grid pattern        a better fit for the structure and location;    -   Not flying 10-20% past the edge on a grid pattern so as to        generate an incomplete point cloud; and    -   Incorrect modification of defaults prior to flight prior to        autonomous operation-speeds, grid overlaps, orbit radius.

The systems and methods have been described herein in detail withrespect to the generation of information about structures of interestvia vehicles incorporating imaging devices. However, the methods andsystems herein can provide additional utility in generating usefulinformation from images of a structure, object, item, person, animal,etc.

With respect to mobile device implementations, the systems and methodsof the present disclosure can be represented by a real time capturefeedback mobile app. These design details indicate for a mobile app butthe concepts apply to other implementations. The capture plan goal inthis scenario can be to capture imagery sufficient to achieve a good 3Dreconstruction of the structure(s) of interest, from which otheranalysis and value can be provided.

Generally, at a base level, when a user is capturing a scene, herequires knowledge of whether the images or video captured will besufficient to generate a good 3D reconstruction of the structure ofinterest. In an implementation, an app on a mobile device can provideinstructions to guide a user to generate a high-quality capture so thatsoftware/algorithms can optimally reconstruct the scene when theinformation generated from the mobile device is suitably transferredtherewith.

Technically, the app configured in a mobile device environment can beconfigured perform at least the following functions:

-   -   Analyze scene texture and ensure there is enough for image        processing algorithms to work;    -   Sense camera movement as combination of rotational and        translational movement; and    -   Provide real time feedback to the user to provide guidance on        speed, movement and contextual capture.

One implementation of a mobile device app can use the OpenCV library toanalyze scene feature content of the video stream and provide basicindications of scene feature tracking quality. The mobile device app canbe configured to capture Inertial Measurement Unit (IMU) data on eachvideo frame and performs motion analysis.

Sample use cases for such a capture feedback mobile device app:

-   -   Flooring estimator uses mobile device to generate a floorplan        for quoting by walking around a room while using the app. The        estimator is notified: (1) if captured imagery is sufficient to        generate complete plan; (2) with special instructions for moving        around furniture; and (3) of potential issues with lack of scene        texture;    -   Homeowner uses mobile device to scan the front of her house to        measure windows for replacement. During the scanning, she        receives information to provide guidance for moving in proper        capture plan and the ability to indicate windows she cares        about;    -   Industrial valve owner uses mobile device to scan entire valve        to send specs to a valve maker for repair, replacement, etc.;    -   Designer uses mobile device to scan existing building exterior        and interior, for input into a SketchUp or Revit model;    -   Homeowner uses mobile device to scan a kitchen for input into a        virtual visualization and product selection engine to see how        products would fit and appear in her home;    -   Roofer travels to a location and uses a mobile device to scan a        roof manually or via unmanned aerial vehicle. Before leaving the        site, she receives notification of proper or improper capture.        The app can provide the ability to check unmanned aerial vehicle        captured imagery or video.

Capture plan goals of the mobile device app can include:

-   -   Ensure that user conducts best possible capture;    -   Indicate predicted quality of output based on scene/capture;    -   Train user to capture videos that yield high-quality        reconstructions; and    -   Integrate into a customer's or 3^(rd) party's mobile app to        manage or direct the capture process.

Features that can be incorporated in the mobile app can include:

-   -   Visual cues for sufficient features, speed, and movement type;    -   Sense rotation-only movement, indicate issues with movement        during image acquisition;    -   Optionally stop capture on pure rotation;    -   Visual cues to direct back to moment/frame of acceptable        movement;    -   Begin seamless record again; and    -   Driven by scene-type specific capture plans.

Scene-types supported by the mobile device can include, for example:

-   -   Whole room interior capture;    -   360° walk-around capture of object (e.g., building or piece of        equipment) that can fit in camera frame;    -   Floor sequential;    -   Façade of structure;    -   Façade plus roof of structure;    -   Whole structure exterior;    -   Whole structure exterior plus interior;    -   Capture of all motion data for back-end usage;    -   Confirm marker found in the scene;    -   Tutorial(s) to provide guidance to user—staged/produced;    -   Live guidance for user generated by real-time image/video        analysis;    -   Accept user input on region of interest, type of capture;    -   Identify items in scene/frame and allow selection;    -   Confirm if scale marker found in the scene, accept specification        of the marker; and    -   Collect data for use in reconstruction—anything that can make        process more efficient, higher quality, more repeatable, etc.

Additional features that can also be included with the mobile app:

-   -   Show live dimensions where marker is in the scene (e.g., live        camera pose estimation);    -   User-selectable region of interest to focus capture;    -   Quality check around certain area(s); and    -   Quick analysis of acquired images or video to determine quality.

Such a mobile app that incorporates the captured image processing,capture plan analysis and improvements, and capture plan goal assessmentcan address at least the following technical challenges that existcurrently:

-   -   Collect and analyze IMU data to determine their usefulness as        input to motion estimation scheme;    -   Collect GPS data and determine their usefulness as input to        motion estimation scheme;    -   Develop framework for combining video, GPS, and IMU data to        track camera motion and produce either basic reconstruction or        reconstruction quality estimate;    -   Track camera movement and detect rotation-only movement;    -   Determine best available movement data (e.g. GPS only        occasionally available);    -   Develop app-accessible, server-based reconstruction quality        predictor or basic reconstruction;    -   Determine how to find last good capture frame, to restart        capture when back in view;    -   Sense/demarcate marker in live video or camera view;    -   Design/implement framework to interface with UI that provides        feedback on condition of scene, movement, etc.; and    -   Capture series of 15-second video captures of scenes that        exemplify the types of movement, texture, etc. that would elicit        feedback from the system. Goal is to use them to explain to UX        team the concepts and for the UX team to illustrate on these        videos the proposed feedback approach.

The user interface and user experience (UI/UX) supports:

-   -   Positive interactive experience to guide the user to an optimal        capture;    -   Adapt experience based on data from framework (movement, scene        quality);    -   Optimize user experience for:        -   display of features in real-time;        -   positive and corrective movement cues;        -   confirmation marker is captured;        -   Series of 15-second video captures of scenes to explain            approach;        -   Select scene type dialogue;        -   Scene-specific directives on movement; and    -   Tutorials—optimized implementation so close at hand but        non-intrusive.

Example scenario(s) for uses of the mobile device app:

-   -   Capture whole room—user has selected whole room scene;    -   User can watch video of example of whole room capture,        indicating where to start, what direction to move, speed to        move, example feedback, etc.;    -   User is directed to start in certain spot, starts capturing, and        is provided with instructions regarding the direction and speed        to move;    -   When system senses very low feature count, e.g., when focused on        white wall, the system can provide instructions to direct user        to high feature area (up to ceiling, down to floor, etc.) to        better ensure continuity of reconstruction;    -   When system senses movement is too fast or too slow instructions        suggestions for speed change can be provided; and    -   When system senses pure rotation, it can provide instructions to        direct user to stop doing it, to move back to position and area        it was last capturing before pure rotation, and direct to start        moving again.

Additional or alternative functionality that can be incorporated intothe mobile app can be lightweight server-based Simultaneous Localizationand Mapping (SLAM) to generate “Quick View”. In this regard, oneimplementation can provide that there is no pure-rotation movement incaptured video, whereas another implementation takes confirmation tonext level by generating a “Quick View” of the captured scenes, usingquick sparse model, for purpose of checking capture quality.

Post capture, the system can be configured to immediately orsubstantially down sample the video, send to server and perform SLAM,then deliver results back to client in a meaningful manner.

In the SLAM approach, it can be important to skip collection of (or nottransmit) rotational movements so that SLAM will complete successfully.

Additional or alternative functionality can include:

-   -   Transfer of SLAM to the mobile device for real-time or        substantially real-time capture quality and sparse scene        building;    -   Optimizes SLAM to run near-real time on phone;    -   Integrates with existing directives and UI interface to maintain        feedback mechanism; and    -   Adds interpolated view in combination with camera pose view.

An expanded UI/UX can include:

-   -   Integrate with existing directives and UI interface to maintain        feedback mechanism;    -   Show real time generation of camera pose view; and    -   Add interpolated view in combination with camera pose view.

Referring next to FIG. 2, shown is an example of a process 200 that canbe used to generate information about structures of interest. Beginningat 203, a capture plan goal can be defined for an imaging event. Thecapture plan goal can be defined by a computer, a user or a combinationthereof. The capture plan goal can be configured to provide one or moredefined information type(s) about a structure of interest, or a part ofa structure of interest. Information type(s) can be generated using oneor more image capture device(s) of a user device or user devices suchas, e.g., an unmanned aerial vehicle, mobile computing device, etc.

At 206, a capture plan can be generated in order to achieve the captureplan goal. The capture plan can be configured to substantially completethe capture plan goal. The capture plan can comprise instructions thatare configured for operating of the user device(s) to implement and/orcomplete the capture plan. The instructions can be associated withoperating the user device(s) and/or navigating the user device(s) to,around, and back from a location proximate to the structure of interest.The capture plan can be generated by the computer and/or the user.

Next, images of the structure or interest or the part of the structureof interest can be acquired by the image capture device(s) at 209. Theimages can be acquired by the user device(s) in a structure imagingevent based on the generated capture plan. For example, an unmannedaerial vehicle can be controlled through, e.g., vehicle operationinstructions, vehicle navigation instructions and/or image acquisitioninstructions to acquire a plurality of images during the structureimaging event. Similarly, operation instructions, navigationinstructions and/or image acquisition instructions can be provided toacquire a plurality of images using other user device(s).

The acquired images can then be processed at 212 to generate informationtypes about the structure of interest. The generated information typescan comprise at least some of the defined information types defined bythe capture plan goal. The defined information types can include a 3Drepresentation comprising a 3D reconstruction or point cloud. Thedefined can also include measurements, counts, identification,orientation, materials or characterizations of of the structure ofinterest or part of the structure of interest.

At 215, the generated information types can be compared to the definedinformation types to determine if they conform to the capture plan goal.If the generated information types satisfy the capture plan goal, thenaccurate information regarding the structure of interest can be providedat 218. If the generated information types are incomplete and thus donot conform, then the process flow can return to 206 to generate asubsequent capture plan to achieve the capture plan goal. For example, asecond capture plan can be generated at 206. In some implementations,the structure of interest (or part of the structure of interest) can besame or different for different capture plans. Reconstructions generatedfrom the imaging events can be compared with information associated withthe real-life structure and/or with each other to provide accuracyinformation associated with the capture plans.

Referring now to FIG. 3, shown is an example of a machine 300 that maybe utilized for the capture planning methodology disclosed herein. Themachine 300 can be a computing device 303 or other processing device,which includes at least one processor circuit, for example, having aprocessor 306 and a memory 309, both of which are coupled to a localinterface 312. To this end, the computing device(s) 303 may comprise,for example, a server computer, mobile computing device (e.g., laptop,tablet, smart phone, etc.) or any other system providing computingcapability. The computing device(s) 303 may include, for example, one ormore display or touch screen devices and various peripheral devices.Even though the computing device 303 is referred to in the singular, itis understood that a plurality of computing devices 303 may be employedin the various arrangements as described above. The local interface 312may comprise, for example, a data bus with an accompanyingaddress/control bus or other bus structure as can be appreciated.

Stored in the memory 309 are both data and several components that areexecutable by the processor 306. In particular, stored in the memory 309and executable by the processor 306 include a capture planningapplication 315 and potentially other applications. Also stored in thememory 309 may be a data store 318 and other data. The data stored inthe data store 318, for example, is associated with the operation of thevarious applications and/or functional entities described below. Forexample, the data store may include databases, object libraries, andother data or information as can be understood. In addition, anoperating system 321 may be stored in the memory 309 and executable bythe processor 306. The data store 318 may be may be located in a singlecomputing device or may be dispersed among many different devices. Thecomponents executed on the computing device 303 include, for example,the capture planning application 315 and other systems, applications,services, processes, engines, or functionality not discussed in detailherein. It is understood that there may be other applications that arestored in the memory 309 and are executable by the processor 306 as canbe appreciated. Where any component discussed herein is implemented inthe form of software, any one of a number of programming languages maybe employed.

The machine 300 can be configured to communicate with one or more userdevice(s) 324 (e.g., an unmanned aerial vehicle, mobile computing deviceor other mobile user device) including an image capture device 327. Forexample, the user device(s) 324 can be communicatively coupled to thecomputing device(s) 303 either directly through a wireless communicationlink or other appropriate wired or wireless communication channel, orindirectly through a network 330 (e.g., WLAN, internet, cellular orother appropriate network or combination of networks). In this way,capture plan information, acquired image information or otherinformation can be communicated between the computing device(s) 303 anduser device(s) 324.

A number of software components are stored in the memory 309 and areexecutable by the processor 306. In this respect, the term “executable”means a program file that is in a form that can ultimately be run by theprocessor 306. Examples of executable programs may be, for example, acompiled program that can be translated into machine instructions in aformat that can be loaded into a random access portion of the memory 309and run by the processor 306, source code that may be expressed inproper format such as object code that is capable of being loaded into arandom access portion of the memory 309 and executed by the processor306, or source code that may be interpreted by another executableprogram to generate instructions in a random access portion of thememory 309 to be executed by the processor 306, etc. An executableprogram may be stored in any portion or component of the memory 309including, for example, random access memory (RAM), read-only memory(ROM), hard drive, solid-state drive, USB flash drive, memory card,optical disc such as compact disc (CD) or digital versatile disc (DVD),floppy disk, magnetic tape, or other memory components.

Also, the processor 306 may represent multiple processors 306 and thememory 309 may represent multiple memories 309 that operate in parallelprocessing circuits, respectively. In such a case, the local interface312 may be an appropriate network that facilitates communication betweenany two of the multiple processors 306, between any processor 306 andany of the memories 309, or between any two of the memories 309, etc.The local interface 312 may comprise additional systems designed tocoordinate this communication, including, for example, performing loadbalancing. The processor 306 may be of electrical or of some otheravailable construction.

Although the capture planning application 315, and other various systemsdescribed herein, may be embodied in software or instructions executedby general purpose hardware as discussed above, as an alternative thesame may also be embodied in dedicated hardware or a combination ofsoftware/general purpose hardware and dedicated hardware. If embodied indedicated hardware, each can be implemented as a circuit or statemachine that employs any one of or a combination of a number oftechnologies. These technologies may include, but are not limited to,discrete logic circuits having logic gates for implementing variouslogic functions upon an application of one or more data signals,application specific integrated circuits having appropriate logic gates,or other components, etc. Such technologies are generally well known bythose skilled in the art and, consequently, are not described in detailherein.

Any logic or application described herein, including the captureplanning application 315, that comprises software or instructions can beembodied in any non-transitory computer-readable medium for use by or inconnection with an instruction execution system such as, for example, aprocessor 306 in a computer system or other system. In this sense, thelogic may comprise, for example, statements including instructions anddeclarations that can be fetched from the computer-readable medium andexecuted by the instruction execution system. The flow diagram of FIG. 2shows an example of the architecture, functionality, and operation ofpossible implementations of a capture planning application 315. In thisregard, each block can represent a module, segment, or portion of code,which comprises one or more executable instructions for implementing thespecified logical function(s). It should also be noted that in somealternative implementations, the functions noted in the blocks may occurout of the order noted in FIG. 2. For example, two blocks shown insuccession in FIG. 2 may in fact be executed substantially concurrentlyor the blocks may sometimes be executed in a different or reverse order,depending upon the functionality involved. Alternate implementations areincluded within the scope of the preferred embodiment of the presentdisclosure in which functions may be executed out of order from thatshown or discussed, including substantially concurrently or in reverseorder, depending on the functionality involved, as would be understoodby those reasonably skilled in the art of the present disclosure.

Communication media appropriate for use in or with the inventions of thepresent disclosure may be exemplified by computer-readable instructions,data structures, program modules, or other data stored on non-transientcomputer-readable media, and may include any information-delivery media.The instructions and data structures stored on the non-transientcomputer-readable media may be transmitted as a modulated data signal tothe computer or server on which the computer-implemented methods of thepresent disclosure are executed. A “modulated data signal” may be asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media may include wired media such asa wired network or direct-wired connection, and wireless media such asacoustic, radio frequency (RF), microwave, infrared (IR) and otherwireless media. The term “computer-readable media” as used herein mayinclude both local non-transient storage media and remote non-transientstorage media connected to the information processors usingcommunication media such as the internet. Non-transientcomputer-readable media do not include mere signals or modulated carrierwaves, but include the storage media that form the source for suchsignals.

In the context of the present disclosure, a “computer-readable medium”can be any medium that can contain, store, or maintain the logic orapplication described herein for use by or in connection with theinstruction execution system. The computer-readable medium can compriseany one of many physical media such as, for example, electronic,magnetic, optical, electromagnetic, infrared, or semiconductor media.More specific examples of a suitable computer-readable medium wouldinclude, but are not limited to, magnetic tapes, magnetic floppydiskettes, magnetic hard drives, memory cards, solid-state drives, USBflash drives, or optical discs. Also, the computer-readable medium maybe a random access memory (RAM) including, for example, static randomaccess memory (SRAM) and dynamic random access memory (DRAM), ormagnetic random access memory (MRAM). In addition, the computer-readablemedium may be a read-only memory (ROM), a programmable read-only memory(PROM), an erasable programmable read-only memory (EPROM), anelectrically erasable programmable read-only memory (EEPROM), or othertype of memory device.

At this time, there is little distinction left between hardware andsoftware implementations of aspects of systems; the use of hardware orsoftware is generally (but not always, in that in certain contexts thechoice between hardware and software can become significant) a designchoice representing cost vs. efficiency tradeoffs. There are variousinformation-processing vehicles by which processes and/or systems and/orother technologies described herein may be implemented, e.g., hardware,software, and/or firmware, and that the preferred vehicle may vary withthe context in which the processes and/or systems and/or othertechnologies are deployed. For example, if an implementer determinesthat speed and accuracy are paramount, the implementer may opt for amainly hardware and/or firmware vehicle; if flexibility is paramount,the implementer may opt for a mainly software implementation; or, yetagain alternatively, the implementer may opt for some combination ofhardware, software, and/or firmware.

The foregoing detailed description has set forth various aspects of thedevices and/or processes for system configuration via the use of blockdiagrams, flowcharts, and/or examples. Insofar as such block diagrams,flowcharts, and/or examples contain one or more functions and/oroperations, it will be understood by those within the art that eachfunction and/or operation within such block diagrams, flowcharts, orexamples can be implemented, individually and/or collectively, by a widerange of hardware, software, firmware, or virtually any combinationthereof. In one embodiment, several portions of the subject matterdescribed herein may be implemented via Application Specific IntegratedCircuits (ASICs), Field Programmable Gate Arrays (FPGAs), digital signalprocessors (DSPs), or other integrated formats. However, those skilledin the art will recognize that some aspects of the aspects disclosedherein, in whole or in part, can be equivalently implemented inintegrated circuits, as one or more computer programs running on one ormore computers, e.g., as one or more programs running on one or morecomputer systems, as one or more programs running on one or moreprocessors, e.g., as one or more programs running on one or moremicroprocessors, as firmware, or as virtually any combination thereof,and that designing the circuitry and/or writing the code for thesoftware and or firmware would be well within the skill of one of skillin the art in light of this disclosure. In addition, those skilled inthe art will appreciate that the mechanisms of the subject matterdescribed herein are capable of being distributed as a program productin a variety of forms, and that an illustrative embodiment of thesubject matter described herein applies regardless of the particulartype of signal bearing medium used to actually carry out thedistribution. Examples of a signal-bearing medium include, but are notlimited to, the following: a recordable type medium such as a floppydisk, a hard disk drive, a CD, a DVD, a digital tape, a computer memory,etc.; and a remote non-transitory storage medium accessed using atransmission type medium such as a digital and/or an analogcommunication medium (e.g., a fiber optic cable, a waveguide, a wiredcommunications link, a wireless communication link, etc.), for example aserver accessed via the internet.

Those skilled in the art will recognize that it is common within the artto describe devices and/or processes in the fashion set forth herein,and thereafter use engineering practices to integrate such describeddevices and/or processes into data-processing systems. That is, at leasta portion of the devices and/or processes described herein can beintegrated into a data processing system via a reasonable amount ofexperimentation. Those having skill in the art will recognize that atypical data processing system generally includes one or more of asystem unit housing, a video display device, a memory such as volatileand non-volatile memory, processors such as microprocessors and digitalsignal processors, computational entities such as operating systems,drivers, graphical user interfaces, and applications programs, one ormore interaction devices, such as a touch pad or screen, and/or controlsystems including feedback loops and control motors, e.g., feedback forsensing position and/or velocity; control motors for moving and/oradjusting components and/or quantities. A typical data processing systemmay be implemented utilizing any suitable commercially availablecomponents, such as those typically found in datacomputing/communication and/or network computing/communication systems.

The herein-described subject matter sometimes illustrates differentcomponents contained within, or connected with, different othercomponents. It is to be understood that such depicted architectures aremerely examples, and that in fact many other architectures can beimplemented which achieve the same functionality. In a conceptual sense,any arrangement of components to achieve the same functionality iseffectively “associated” such that the desired functionality isachieved. Hence, any two components herein combined to achieve aparticular functionality can be seen as “associated with” each othersuch that the desired functionality is achieved, irrespective ofarchitectures or intermedial components. Likewise, any two components soassociated can also be viewed as being “operably connected”, or“operably coupled”, to each other to achieve the desired functionality,and any two components capable of being so associated can also be viewedas being “operably couplable”, to each other to achieve the desiredfunctionality. Specific examples of operably couplable include but arenot limited to physically mateable and/or physically interactingcomponents and/or wirelessly interactable and/or wirelessly interactingcomponents and/or logically interacting and/or logically interactablecomponents.

As described herein, the exemplary aspects have been described andillustrated in the drawings and the specification. The exemplary aspectswere chosen and described in order to explain certain principles of theinvention and their practical application, to thereby enable othersskilled in the art to make and utilize various exemplary aspects of thepresent invention, as well as various alternatives and modificationsthereof. As is evident from the foregoing description, certain aspectsof the present invention are not limited by the particular details ofthe examples illustrated herein, and it is therefore contemplated thatother modifications and applications, or equivalents thereof, will occurto those skilled in the art. Many changes, modifications, variations andother uses and applications of the present construction will, however,become apparent to those skilled in the art after considering thespecification and the accompanying drawings. All such changes;modifications, variations and other uses and applications which do notdepart from the spirit and scope of the invention are deemed to becovered by the invention which is limited only by the claims whichfollow.

What is claimed is:
 1. A method, comprising: a. defining, by a computeror a user, a capture plan goal for a structure imaging event, whereinthe capture plan goal is configured to provide one or more informationtypes defined by the capture plan goal about a structure of interest,and wherein the one or more defined information types are generated viaaerial imaging of the structure of interest by an unmanned aerialvehicle configured with at least one image capture device; b.generating, by the computer or the user, a first capture plan configuredto substantially complete the capture plan goal, wherein the firstcapture plan comprises instructions configured for operating of theunmanned aerial vehicle, wherein the instructions are associated withoperating the unmanned aerial vehicle and navigating, by the computer orthe user, the unmanned aerial vehicle to, around, and back from alocation proximate to the structure of interest; c. acquiring, by theunmanned aerial vehicle, a plurality of images of the structure ofinterest in a first structure imaging event, wherein the plurality ofacquired images are acquired by the unmanned aerial vehicle during thefirst structure imaging event according to: i. vehicle operationinstructions; ii. vehicle navigation instructions; and iii. imageacquisition instructions; d. processing, by the computer, the pluralityof acquired images to generate information types about the structure ofinterest, wherein the generated information types comprise at least someof the one or more information types about the structure of interestdefined by the capture plan goal; e. comparing, by the computer or theuser, each of the generated information types with the one or moredefined information types; and f. generating feedback about whether allor part of the capture plan goal has been achieved in the firststructure imaging event, wherein the feedback is provided to thecomputer or to the user, and wherein the feedback is optionally used inthe generation of a second capture plan.
 2. The method of claim 1,wherein the feedback comprises information about one or more of thefollowing: a. view angle derivable from an acquired image of thestructure of interest or structure part and a corresponding surface orsurface part; b. angle of triangulation derivable from each of twopoints in two images of the same structure of interest or structurepart; c. ground sample distance (“GSD”) between the unmanned aerialvehicle and the structure of interest or structure part during the imageacquisition; and d. structure sample distance (“SSD”) derivable for theunmanned aerial vehicle and the structure of interest or part during theimage acquisition.
 3. The method of claim 1, wherein instructions areoptionally provided for imaging of at least part of the structure ofinterest from ground-level.
 4. The method of claim 1, wherein at leastsome of the image processing is conducted during the first structureimaging event, and wherein at least some feedback is incorporated in thevehicle operation instructions, the vehicle navigation instructions, orthe image acquisition instructions, thereby allowing modification of atleast some of the first capture plan during the first structure imagingevent.
 5. A method, comprising: a. defining, by a computer or a user, acapture plan goal for an aerial imaging event, wherein the capture plangoal is configured to provide one or more defined information typesabout a structure of interest, and wherein the one or more definedinformation types are generated via aerial imaging of the structure ofinterest by an unmanned aerial vehicle configured with at least oneimage capture device; b. generating, by the computer or the user, afirst capture plan configured to substantially complete the capture plangoal, wherein the first capture plan comprises instructions configuredfor operating of the unmanned aerial vehicle, wherein the instructionsare associated with operating the unmanned aerial vehicle andnavigating, by the computer or the user, the unmanned aerial vehicle to,around, and back from a location proximate to the structure of interest;c. acquiring, by the unmanned aerial vehicle, a plurality of images ofthe structure of interest in a first structure imaging event, whereinthe images are acquired by the unmanned aerial vehicle during the firststructure imaging event according to: i. vehicle operation instructions;ii. vehicle navigation instructions; and iii. image acquisitioninstructions; d. processing, by the computer, the plurality of acquiredimages to generate information types about the structure of interest,wherein the generated information types comprise at least some of theone or more defined information types about the structure of interestdefined by the capture plan goal; e. generating a second capture planfor use in a second structure imaging event, wherein: i. the structureof interest imaged in the first structure imaging event and a structureof interest imaged in the second structure imaging event are the same ordifferent; ii. an output of each of the first and second structureimaging events is a 3D reconstruction of the structure of interestimaged in the first or the second structure imaging event; and f.comparing each of a first 3D reconstruction generated from the firststructure imaging event and a second 3D reconstruction generated fromthe second structure imaging event with information associated with anassociated real-life structure, thereby providing information associatedwith the first and second 3D reconstructions.
 6. The method of claim 5,wherein the information associated with the first and second 3Dreconstructions incorporates measurement information providing a percenterror or confidence level that the first and second 3D reconstructionshave the same features or dimensions as the associated real-lifestructure.
 7. The method of claim 5, wherein the information associatedwith the first and second 3D reconstructions is incorporated into acapture plan used in subsequent structure imaging events.
 8. The methodof claim 5, wherein the information associated with the first 3Dreconstruction is incorporated into a design of the second capture plan.9. The method of claim 1, comprising determining, by the computer or theuser, whether some or all of the generated information typessubstantially align with each of the one or more information typesdefined by the capture plan goal.
 10. The method of claim 1, wherein theone or more defined information types comprises one or more of: a. a 3Drepresentation of all or part of the structure of interest, wherein the3D representation comprises a 3D reconstruction or a point cloud; b.measurements of all or part of the structure of interest; c. counts ofthe structure of interest or parts of the structure of interest; d.identification of the structure of interest or parts of the structure ofinterest; e. orientation of two objects on or near the structure ofinterest with respect to each other in a scene in which the structure ofinterest is located; f. identification of materials incorporated in thestructure of interest; and g. characterization of a condition state forthe structure of interest or parts of the structure of interest.
 11. Themethod of claim 1, wherein the first capture plan incorporatesinformation comprising at least some of: a. information about thestructure of interest known prior to the image acquisition, wherein theknown structure information comprises one or more of: i. estimateddimensions of all or part of the structure of interest; ii. GPS locationof the structure of interest; iii. estimated height of the structure ofinterest; iv. estimated outer boundaries of the structure of interest;and v. obstacles proximate to the structure of interest; b. unmannedaerial vehicle and image capture device information comprising one ormore of: i. image capture device lens resolution; ii. unmanned aerialvehicle battery life; iii. inertial measurement sensors and associatedcomponentry; iv. unmanned aerial vehicle GPS status or interferenceduring the image acquisition; v. altitude of the unmanned aerial vehicleduring the image acquisition; vi. temperature data; and vii. unmannedaerial vehicle clock data; c. number of images to be acquired during thefirst structure imaging event; d. number of images to be acquired perunit time during the first structure imaging event; e. number of imagesto be acquired per unit of distance traveled by the unmanned aerialvehicle during the first structure imaging event; f. distances betweenthe unmanned aerial vehicle and all or part of the structure of interestduring the image acquisition; g. view angle derivable from an acquiredimage of the structure of interest or structure part and a correspondingsurface or surface part; h. angle of triangulation derivable from eachof two points in two images of the same structure of interest orstructure part; i. structure sample distance (“SSD”) between theunmanned aerial vehicle and the structure of interest or structure partduring the image acquisition; j. ground sample distance (“GSD”) betweenthe unmanned aerial vehicle and the structure of interest or structurepart during the image acquisition; k. speed at which the unmanned aerialvehicle is to be moving in the scene or environment during the imageacquisition; and l. number of passes to be made by the unmanned aerialvehicle in and around the structure of interest or parts of thestructure during the image acquisition.
 12. The method of claim 1,wherein the method further comprises generating information about one ormore of: a. resolution of the plurality of acquired images; b. presenceor absence of occlusions in the plurality of acquired images; c.potential error range of information derived from the plurality ofacquired images; d. information associated with weather and illuminationaround the structure of interest during the image acquisition; e.orientation of the imaging device with respect to sunlight directionduring the image acquisition; f. unmanned aerial vehicle gimbal positionand stability during image acquisition; g. obstructions proximate to thestructure of interest during the image acquisition; or h. acquired imagecharacteristics associated with navigation of the unmanned aerialvehicle, wherein the acquired image characteristics result at least inpart from operation of the unmanned aerial vehicle according to thefirst capture plan.
 13. The method of claim 12, wherein the unmannedaerial vehicle operations comprise one or more of: a. the degree ofalignment of the unmanned aerial vehicle with the all or part of thestructure of interest during the image acquisition; b. the degree ofoverlap between the acquired images incorporating an interior of thestructure of interest and images incorporating one or more boundaries ofthe structure of interest; c. the degree of centering of the unmannedaerial vehicle relative to the structure of interest during the imageacquisition; d. degree of forward and side overlap between the acquiredimages when the first capture plan is configured to acquire images in agrid pattern relative to the structure of interest; e. degree of overlapof image radii between acquired images when the first capture plan isconfigured to acquire images in a circular pattern relative to thestructure of interest; f. yaw of the unmanned aerial vehicle during theimage acquisition; and g. orientation of the at least one image capturedevice relative to the structure of interest during the imageacquisition.
 14. The method of claim 1, wherein the one or more definedinformation types comprise one or more measurements of the structure ofinterest, and generated roof dimensions are within about 5% of actualroof dimensions when the actual roof dimensions are directly measured.15. The method of claim 1, wherein if one or more of the generatedinformation types do not substantially conform to the one or moredefined information types defined by the capture plan goal, the methodfurther comprises: a. generating a second capture plan incorporatinginformation derived from processing of the plurality of acquired imagesfrom the first structure imaging event, wherein the second capture planis used in a second structure imaging event.
 16. The method of claim 15,wherein a structure of interest in the second imaging event is the sameas the structure of interest in the first imaging event.
 17. The methodof claim 15, wherein a structure of interest in the second imaging eventis different from the structure of interest in the first imaging event.18. The method of claim 10, wherein the 3D reconstruction is generated,the 3D reconstruction incorporating all or part of the structure ofinterest, and information associated with the 3D reconstruction isprovided, wherein the provided information comprises one or more of: a.point cloud density when the 3D reconstruction comprises a point cloud;b. re-projection error measurement for the 3D reconstruction; and c.accuracy indication for the 3D reconstruction, wherein the accuracyindication is provided in the form of a probability or percentage thatthe 3D reconstruction is an accurate representation of all or part ofthe structure.
 19. The method of claim 10, wherein the 3D reconstructionis generated, the 3D reconstruction comprises a wireframe, and whereinthe wireframe comprises all or part of the structure of interest, themethod further comprising: a. evaluating the wireframe to identifymissing or occluded areas; and b. analyzing the plurality of acquiredimages from which the wireframe was derived to provide informationassociated with a diagnosis of one or more reasons for the presence ofthe missing or occluded areas.
 20. The method of claim 19, furthercomprising incorporating the provided information associated with thediagnosis in a second capture plan.